Detecton of Cancer Metastass Usng a Novel Macroscopc Hyperspectral Method Hamed Akbar a, Luma V. Halg a, Hongzheng Zhang b, Dongsheng Wang b, Zhuo Georga Chen b, Baowe Fe a,c,d,* a Department of Radology and Imagng Scences, b Department of Hematology and Medcal Oncology, c Wnshp Cancer Insttute, d Department of Bomedcal Engneerng, Emory Unversty and Georga Insttute of Technology, Atlanta, GA * E-mal: bfe@emory.edu; Webste: http://felab.org ABSTRACT The proposed macroscopc optcal hstopathology ncludes a broad-band lght source whch s selected to llumnate the tssue glass slde of suspcous pathology, and a hyperspectral camera that captures all wavelength bands from 450 to 950 nm. The system has been traned to classfy each hstologc slde based on predetermned pathology wth lght havng a wavelength wthn a predetermned range of wavelengths. Ths technology s able to capture both the spatal and spectral data of tssue. Hghly metastatc human head and neck cancer cells were transplanted to nude mce. After 2-3 weeks, the mce were euthanzed and the lymph nodes and lung tssues were sent to pathology. The metastatc cancer s studed n lymph nodes and lungs. The pathologcal sldes were maged usng the hyperspectral camera. The results of the proposed method were compared to the pathologc report. Usng hyperspectral mages, a lbrary of spectral sgnatures for dfferent tssues was created. The hgh-dmensonal data were classfed usng a support vector machne (SVM). The spectra are extracted n cancerous and non-cancerous tssues n lymph nodes and lung tssues. The spectral dmenson s used as the nput of SVM. Twelve glasses are employed for tranng and evaluaton. The leave-one-out cross-valdaton method s used n the study. After tranng, the proposed SVM method can detect the metastatc cancer n lung hstologc sldes wth the specfcty of 97.7% and the senstvty of 92.6%, and n lymph node sldes wth the specfcty of 98.3% and the senstvty of 96.2%. Ths method may be able to help pathologsts to evaluate many hstologc sldes n a short tme. Keywords: Cancer detecton, hyperspectral magng, Head and neck cancer, Optcal magng, Infrared magng, Support vector machne 1. INTRODUCTION The current gold standard for dagnoss of cancer s bopsy, whch nvolves the removal of tssue samples for examnaton. Laboratory results for the determnaton of hstopathology of a suspected tumor may generally take several days [1]. Hyperspectral mage data provde a powerful tool for non-nvasve tssue analyses. Ths technology s capable of capturng both the spatal and spectral nformaton of an organ n one snapshot. In fact, a hyperspectral magng system produces several narrow band mages at dfferent wavelengths. Compared to conventonal color cameras and other flter-based magng systems, ths system produces full neghborng spectral data wth spectral and spatal nformaton [2]. Metastatc tumor or a metastass s one of the most mportant concerns n cancer. Hyperspectral magng (HSI) can extend the human vson to near-nfrared and nfrared wavelength regons. HSI has already been appled n the medcal feld [1, 3-14]. HSI s appled to provde quanttatve data about the tssue oxygen saturaton n patents wth perpheral vascular dsease, to detect schemc regons of the ntestne durng surgery, to predct and follow healng n foot ulcers of dabetc patents, and to dagnose hemorrhagc shock [15]. The hyperspectral magng that operates as a tunable optcal band pass flter has been utlzed to dscrmnate between non-cancerous and cancerous tssue [16]. Recently, Fourer transform nfrared (FTIR) spectroscopy has wdely been used as a cancer dagnostc technque that can only be appled to pont measurements [17-22]. The hyperspectral magng was evaluated for cytologc dagnoss of cancer cells. HSI was employed to obtan hyperspectrum from a normal human fbroblast, as well as ts telomerase-mmortalzed and SV40-transformed dervatves. Novel algorthms were developed to dfferentate among these cell models based on spectral and spatal dfferences [23]. The hgh-resoluton hyperspectral magng mcroscopy was evaluated to detect Medcal Imagng 2012: Bomedcal Applcatons n Molecular, Structural, and Functonal Imagng, edted by Robert C. Molthen, John B. Weaver, Proc. of SPIE Vol. 8317, 831711 2012 SPIE CCC code: 1605-7422/12/$18 do: 10.1117/12.912026 Proc. of SPIE Vol. 8317 831711-1
abnormaltes n skn tssue usng Hematoxyln and Eosn staned preparatons of normal and abnormal skn, bengn nev and melanomas [6]. The capablty of ths method was demonstrated by observng gastrc tumors n 10 human subjects. The spectral sgnatures of the gastrc cancer and non-cancerous stomach tssue were created n nfrared wavelengths. The hyperspectral mages were processed va the standard devaton of the spectral dagram, support vector machne, and frst dervatves and ntegral of n spectral dagram were proposed to enhance and detect the cancerous regons [24]. So far, none of the research has used hyperspectral magng for cancer detecton n pathologcal sldes. The goal of ths study s to test the potental of usng hyperspectral magng and advance mage analyss methods for cancer detecton. Fgure 1. A schematc vew of hyperspectral mages (left). Rght: The spectral graphs of two pxels from the hstologc slde of lymph node tssue. The graph depcts the ntenstes for each pxel n that wavelength. Fgure 2. The mean of spectral sgnatures of the orgnal cancerous tssues and the metastatc tumor n mce. The horzontal axs shows dfferent wavelengths n nanometers. The vertcal axs shows the ntenstes. The orgnal tumor pxels are shown wth the red lne; the metastatc tumor n the lymph nodes are shown n the blue lne wth red squares; and the metastatc tumor n the lungs are shown n the blue lne. 2. METHODS The hyperspectral mages were captured of three normal and three pathologc sldes for each organ,.e. lung and lymph node, usng a hyperspectral camera. Each pxel n a hyperspectral mage has dfferent ntenstes at dfferent Proc. of SPIE Vol. 8317 831711-2
wavelengths. These ntenstes make the spectral dagram or the spectral sgnature for each pxel. Fgure 1 shows a schematc vew of the hyperspectral mage. We made a spectra lbrary by capturng spectral dagrams of the orgnal cancerous tssue and the metastatc tumor n the lungs and lymph nodes. Fgure 2 shows the ntensty spectra of the head and neck tumor and ts metastass to the lungs and the lymph nodes. The defned regons were utlzed to tran the support vector machne (SVM) to classfy the normal and malgnant regons. Usng the traned SVM, each tssue slde was analyzed and classfed. The SVM was traned and evaluated wth the leave-one-out cross-valdaton method [25, 26]. Fgure 3 shows the flowchart of the method. HSI of the New HSI of a Fgure 3. The flowchart of the method. The head and neck tumor was transplanted nto 19 nude mce. The cell lne that was used n ths study was 686LN- M4e. Ths cell lne s a hghly metastatc, human head and neck cancer cell lne. After 2-3 weeks, the mce were euthanzed and the lymph nodes and lung tssues were sent to the pathology faclty at our nsttuton. The metastatc cancer s studed n lymph nodes and lungs. The pathologcal sldes were maged usng the hyperspectral camera. To capture the hyperspectral mage data, a CR magng system system (Calper, Hopknton, MA) was used. A successful approach to HSI data classfcaton s based on the use of multlayer perceptrons (MLP) and radal bass functon neural networks (RBFNNs). However, these approaches are not effectve when dealng wth a hgh number of spectral bands, snce they are hghly senstve to the Hughes phenomenon. In the recent years, support vector machnes (SVMs) have been successfully used for HSI data classfcaton. The SVMs can effcently handle large nput spaces or nosy samples, and produce sparse solutons [27]. Therefore, to classfy the compressed data and to segment the hyperspectral mage, support vector machne (SVM) s chosen for ths study. 2.1 CR Maestro systems CR Maestro systems can acqure full hyperspectral data wth down to 25 mcron/pxel resoluton n only a few seconds. Insde the Maestro system s a lqud crystal tunable flter that utlzes a sold-state lqud-crystal desgn that blocks unwanted wavelengths and transmts the requred wavelength. It changes wavelengths n mllseconds wthout nose or vbraton. The CR's spectral magng system generates x-number of mages, where x s user-defned. Each mage set contans the measurement of the spectrum of all the ponts whch comprse t. The wavelength range of nterest was defned between 450-950 nm wth 5 nm ncrements. The seres or spectral cube contans spectra from 450 to 950 nm, wth each mage contanng 1.4 mllon data ponts. The exctaton lght was set on the whte llumnaton plus the nfrared nternal llumnaton and no emsson flter was used. Each pxel n the hyperspectral mage has a sequence of reflectance at dfferent spectral wavelengths that can dsplay the spectral sgnature of that pxel. 2.2 Least Squares SVMs In Support Vector Machnes (SVMs), a convex quadratc programmng (QP) solves the classfcaton problem. Suykens and Vandewalle proposed a new verson of SVM classfers, whch was named Least Squares SVMs (LS-SVMs) [28]. In LS-SVMs nstead of nequalty constrants, a two-norm wth equalty are appled. Therefore, nstead of a QP problem n dual space, a lnear set of equatons are obtaned. The SVM tres to fnd a large margn for classfcaton. However, the LS-SVM that s used n ths paper looks for a rdge regresson for classfcaton wth bnary targets. Ths method overcomes some dsadvantages of SVM. For example the selecton of hyperparameters s not as problematc. The sze of the matrx nvolved n the QP problem s also drectly proportonal to the number of tranng ponts [29]. Suykens and Vandewalle modfed the Vapnk s optmzaton functon of the SVM as follows [28]: Proc. of SPIE Vol. 8317 831711-3
1 1 ( γ e (1) N T Mn f w, e) = w w + w, b, e 2 2 = 1 subject to the equalty constrants T y[ w ϕ ( x ) + b] = 1 e, = 1,..., N (2) where w s the weghtng vector, b s the bas term, e s for msclassfcatons, and γ s the tunng parameter. Ths constraned optmzaton problem can be solved by determnng the saddle ponts n the Lagrange functonal as, N = 1 2 T L( w, b, e; α ) = f ( w, b, e) α { y [ w ϕ( x ) + b] 1+ e } (3) where α R are Lagrange multplers that can be postve or negatve n the LS-SVM formulaton. In SVMs, t s possble to choose several types of kernel functons ncludng lnear, polynomal, radal bass functon (RBF), multlayer perceptron (MLP) wth one hdden layer and splnes. The RBF kernel s used n ths paper as follows: K( x, x) = exp{ x x σ } (4) where σ s constant. Spectral data of each pxel n the hyperspectral mage was employed as the nput of the SVM, and the output s ether cancerous or non-cancerous pxel. In each slde, the regons that are defntely normal or malgnant were selected and used for SVM tranng and evaluaton. These regons were defned based on pathologcal reports. The leave-one-out cross-valdaton method s utlzed n ths study. Here we have nput vectors of 101 elements n 5 nm spectral resoluton mages. 2 2 Fgure 4. The mean spectral sgnatures of the cancerous and normal tssue n mce. The horzontal axs shows dfferent wavelengths n nanometers, and the vertcal axs shows the ntensty. Tumor pxels are shown n the redsquared lne, and the normal tssue pxels are shown n the blue lne. Left panel shows the lymph node; and the rght panel shows the lung tssue. 2.3 Evaluaton Crtera The method was evaluated based on pxel detecton. Senstvty and specfcty were used as statstcal measures of the performance of the bnary classfcaton method [30-33]. Senstvty measures the proporton of actual postves whch are correctly dentfed as postve,.e. the percentage of tumor pxels whch are correctly dentfed as tumor tssue. The senstvty expresses as follows: Senstvty TP TP + FN =, (5) Proc. of SPIE Vol. 8317 831711-4
where TP and FN are true postve and false negatve, respectvely. When a pxel was not detected as a tumor pxel, the detecton was consdered as a false negatve f the pxel was ndeed a tumor pxel n the manually created map. When a pxel was detected as a tumor tssue, the detecton was a false postve f the pxel was not tumor tssue. Specfcty measures the proporton of negatves whch are correctly dentfed,.e., the percentage of healthy tssue correctly dentfed as not havng cancer. The followng equaton shows the specfcty calculaton: TN Specfcty =, (6) TN + FP where TN and FP are true negatve and false postve, respectvely. The FNR was defned as the number of false negatve pxels dvded by the total number of the tumor pxel. When a pxel was detected as tumor tssue, the detecton was false postve f the pxel was not a tumor. The FPR was defned as the number of false postve pxels dvded by the total number of normal tssue. In each pathologcal slde, the regon that s defntely malgnant or normal tssue was chosen. The evaluaton crtera were calculated based on these regons. Fgure 5. Cancer detecton results n pathologcal sldes for lung tssue (top) and lymph node tssue (bottom). From left to rght, dgtal mages by a mcroscope; RGB mages that are made by usng three bands of the hyperspectral mage; the bnary results, and the overlad mages where the green regons show the tumor. 3. RESULTS The method was evaluated by data sets of three sldes of normal lung tssue, three sldes of cancerous lung tssue, three sldes of normal lymph node tssue, and three sldes of cancerous lymph node tssue. Fg. 5 shows cancer detecton results n the lung and at the lymph node n the pathologcal sldes. The detecton performance was evaluated wth respect to the pathologcal report. The numercal results of the senstvty and specfcty are shown n Table 1. Table 1. Quanttatve evaluaton results of the pxel-by-pxel classfcaton. Specfcty (%) Senstvty (%) FPR (%) FNR (%) Lymph nodes 98.3 96.2 1.7 3.8 Lungs 97.7 92.6 2.3 7.4 Proc. of SPIE Vol. 8317 831711-5
4. DISCUSSION AND CONCLUSION Hyperspectral magng and advance mage analyss methods are proposed to ad the detecton of cancer n pathologc sldes. Hyperspectral mage can help pathologst to detect the cancer wthout consumng long tme for each slde and can be used not only for dagnoss but also for determnng the tumor metastass after bopsy. Ths technology expands the vson from the three RGB spectral bands to more than a hundred spectral bands. The large number of data n hyperspectral mages can be processed to broaden the spectral range and can supply useful nformaton for medcal doctors. The proposed method detects a cancerous tssue pxel by pxel. The detecton of one pxel as a cancerous pxel does not depend on adjacent pxels. Therefore, the cancer sze or shape would not cause a problem. Ths method could detect the cancerous tssue n the lung and the lymph node wth hgh specfcty and senstvty. Hyperspectral magng can be used as a supportng tool for pathologst to evaluate a large number of sldes n a short tme. Hstopathology s the current gold standard for dagnoss of cancer. However, even ths method has problems such as nvasveness, great dependence on the judgment of pathologsts, and needs tme for results preparng. Moreover, the bopsy specmens can only be captured from a few ponts. A smple, nonnvasve, and relable technque that enables rapd detecton of cancer would ad many physcans. ACKNOWLEDGEMENT Ths research s supported n part by NIH grant R01CA156775 (PI: Fe), Coulter Translatonal Research Grant (PIs: Fe and Hu), Georga Cancer Coalton Dstngushed Clncans and Scentsts Award (PI: Fe), Emory Molecular and Translatonal Imagng Center (NIH P50CA128301), SPORE n Head and Neck Cancer (NIH P50CA128613), and Atlanta Clncal and Translatonal Scence Insttute (ACTSI) that s supported by PHS Grant UL1 RR025008 from the Clncal and Translatonal Scence Award program. REFERENCES [1] S. G. Kong, E. M. Martn, and T. Vo-Dnh, Hyperspectral Fluorescence Imagng for Mouse Skn Tumor Detecton, ETRI Journal, 28(6), 770-776 (2006). [2] H. Akbar, Y. Kosug, K. Kojma et al., Detecton and analyss of the ntestnal schema usng vsble and nvsble hyperspectral magng, IEEE Trans Bomed.Eng., 57(8), 2011-2017 (2010). [3] L. C. Canco, A. I. Batchnsky, J. R. Mansfeld et al., Hyperspectral magng: A new approach to the dagnoss of hemorrhagc shock, Journal of Trauma-Injury Infecton and Crtcal Care, 60(5), 1087-1095 (2006). [4] S. L. Best, A. Thapa, M. S. Holzer et al., "Assessment of Renal Oxygenaton Durng Partal Nephrectomy Usng DLP Hyperspectral Imagng, In Book Emergng Dgtal Mcromrror Devce Based Systems and Applcatons I Edted by Douglass MR and Oden PI, 7932, (2011). [5] M. Crow, S. Marnakos, A. Chlkot et al., "Smultaneous Molecular Imagng of EGFR and HER2 Usng Hyperspectral Darkfeld Mcroscopy and Immunotargeted Nanopartcles." Conference on Plasmoncs n Bology and Medcne VI, 7192 (2009). [6] D. T. Dcker, J. Lerner, P. Van Belle et al., Dfferentaton of normal skn and melanoma usng hgh resoluton hyperspectral magng, Cancer Bology & Therapy, 5(8), 1033-1038 (2006). [7] N. G. Dolloff, X. H. Ma, D. T. Dcker et al., Spectral magng-based methods for quantfyng autophagy and apoptoss, Cancer Bology & Therapy, 12(4), 349-356 (2011). [8] H. Erves, and N. B. Targhetta, Implementaton of a 3-D Hyperspectral Instrument for Skn Imagng Applcatons, IEEE Transactons on Instrumentaton and Measurement, 58(3), 631-638 (2009). [9] M. Ishhara, M. Sato, K. Matsumura et al., "Development of the hyperspectral cellular magng system to apply to regeneratve medcne," Optcs n Tssue Engneerng and Regeneratve Medcne Iv,7566 (2010). [10] R. Jolvot, H. Nugroho, P. Vabres et al., "Valdaton of a 2D multspectral camera: applcaton to dermatology/cosmetology on a populaton coverng fve skn phototypes." Conference on Clncal and Bomedcal Spectroscopy and Imagng II, 8087, (2011). [11] A. H. Kashan, E. Krkman, G. Martn et al., Hyperspectral Computed Tomographc Imagng Spectroscopy of Vascular Oxygen Gradents n the Rabbt Retna In Vvo, Plos One, 6(9), (2011). Proc. of SPIE Vol. 8317 831711-6
[12] D. C. Kellcut, J. M. Weswasser, S. Arora et al., Emergng Technology: Hyperspectral Imagng, Perspectves n Vascular Surgery and Endovascular Therapy, 16(1), 53-57 (2004). [13] R. T. Kester, L. Gao, N. Bedard et al., "Real-Tme Hyperspectral Endoscope for Early Cancer Dagnostcs," Advanced Bomedcal and Clncal Dagnostc Systems V, 7555, (2010). [14] S. G. Kong, Z. Du, M. Martn et al., "Hyperspectral fluorescence mage analyss for use n medcal dagnostcs," Advanced Bomedcal and Clncal Dagnostc Systems III, 5692, (2005). [15] h. Akbar, Y. Kosug, K. Kojma et al., Hyperspectral Image Segmentaton and ts Applcaton n Abdomnal Surgery, Internatonal Journal of Functonal Informatcs and Personalsed Medcne, 2(2), 201-216 (2009). [16] M. E. Martn, M. B. Wabuyele, K. Chen et al., Development of an advanced hyperspectral magng (HSI) system wth applcatons for cancer detecton, Annals of Bomedcal Engneerng, 34(6), 1061-1068 (2006). [17] N. Fujoka, Y. Mormoto, T. Ara et al., Dscrmnaton between normal and malgnant human gastrc tssues by Fourer transform nfrared spectroscopy, Cancer Detecton and Preventon, 28(1), 32-36 (2004). [18] A. Tannapfel, S. Schmelzer, M. Bencke et al., Expresson of the p53 homologues p63 and p73 n multple smultaneous gastrc cancer, Journal of Pathology, 195(2), 163-170 (2001). [19] M. A. Cohenford, T. A. Godwn, F. Cahn et al., Infrared Spectroscopy of Normal and Abnormal Cervcal Smears: Evaluaton by Prncpal Component Analyss, Gynecologc Oncology, 66(1), 59-65 (1997). [20] P. T. T. Wong, S. M. Goldsten, R. C. Grekn et al., Dstnct Infrared Spectroscopc Patterns Of Human Basal- Cell Carcnoma Of The Skn, Cancer Research, 53(4), 762-765 (1993). [21] B. Rgas, and P. T. T. Wong, Human Colon Adenocarcnoma Cell-Lnes Dsplay Infrared Spectroscopc Features Of Malgnant Colon Tssues, Cancer Research, 52(1), 84-88 (1992). [22] L. M. McIntosh, M. Jackson, H. H. Mantsch et al., Infrared spectra of basal cell carcnomas are dstnct from non-tumor-bearng skn components, Journal of Investgatve Dermatology, 112(6), 951-956 (1999). [23] A. M. Sddq, H. L, F. Faruque et al., Use of hyperspectral magng to dstngush normal, precancerous, and cancerous cells, Cancer Cytopathology, 114(1), 13-21 (2008). [24] H. Akbar, K. Uto, Y. Kosug et al., Cancer detecton usng nfrared hyperspectral magng, Cancer Scence, 102(4), 852-857 (2011). [25] X. F. Yang, D. Schuster, V. Master, P. Neh, A. Fenster, B.W. Fe, "Automatc 3D Segmentaton of Ultrasound Images Usng Atlas Regstraton and Statstcal Texture Pror," Proceedngs of SPIE Medcal Imagng: Vsualzaton Image-Guded Procedures and Modelng, 7964, 796432-1~8 (2011). [26] H. Akbar, X. Yang, L. Halg, B.W. Fe, "3D Segmentaton of Prostate Ultrasound Images usng Wavelet Transform." Proceedngs of SPIE Medcal Imagng - Image Processng, 7962, 79622K-1~8 (2011) [27] G. Camps-Valls, and L. Bruzzone, Kernel-based methods for hyperspectral mage classfcaton, IEEE Transactons on Geoscence and Remote Sensng, 43(6), 1351-1362 (2005). [28] J. A. K. Suykens, and J. Vandewalle, Least Squares Support Vector Machne Classfers, Neural Processng Letters, 9(3), 293-300 (1999). [29] T. van Gestel, J. A. K. Suykens, B. Baesens et al., Benchmarkng least squares support vector machne classfers, Machne Learnng, 54(1), 5-32 (2004). [30] H. Wang, D. Feyes, J. Mulvhl et al., "Multscale fuzzy C-means mage classfcaton for multple weghted MR mages for the assessment of photodynamc therapy n mce," Proceedngs of SPIE Medcal Imagng: Image Processng, 6512, 65122W-1~9. (2007). [31] H. S. Wang, and B. W. Fe, A modfed fuzzy C-means classfcaton method usng a multscale dffuson flterng scheme, Medcal Image Analyss, 13(2), 193-202 (2009). [32] X. F. Yang, and B. W. Fe, A multscale and multblock fuzzy C-means classfcaton method for bran MR mages, Medcal Physcs, 38(6), 2879-2891 (2011). [33] K. Boge, X. Wang, B. Fe et al., New technque for real-tme nterface pressure analyss: gettng more out of large mage data sets., J Rehabl Res Dev, 45(4), 523-35 (2008). Proc. of SPIE Vol. 8317 831711-7
Hamed Akbar, Luma V. Halg, Hongzheng Zhang, Dongsheng Wang, Zhuo G. Chen and Baowe Fe, "Detecton of cancer metastass usng a novel macroscopc hyperspectral method", Proc. SPIE 8317, 831711 (2012) Copyrght 2012 Socety of Photo-Optcal Instrumentaton Engneers (SPIE). One prnt or electronc copy may be made for personal use only. Systematc reproducton and dstrbuton, duplcaton of any materal n ths paper for a fee or for commercal purposes, or modfcaton of the content of the paper are prohbted. http://dx.do.org/10.1117/12.912026