Biomedical Engineering in the Metroplex: Hyperspectral Imaging for Medical Use

Transcription

1 Biomedical Engineering in the Metroplex: Hyperspectral Imaging for Medical Use Edward H. Livingston, MD, FACS, AGAF Professor and Chairman-Gastrointestinal and Endocrine Surgery Professor and Chairman-Graduate Program in Biomedical Engineering University of Texas Southwestern School of Medicine Contributing Editor-JAMA 3/13/2012 1

2 The Problem Surgeons cannot see what they are doing when they operate Does this worry you? (it should) 3/13/2012 2

3 The Problem Gallbladder Disease is one of the most frequent causes for hospitalizations and surgery 750,000 hospital admissions annually >500,000 Cholecystectomies performed annually % Bile Duct Injury Result in up to 2,500 injuries per year.

4 What Does The Gallbladder Do? 3/13/2012 4

5

6 What Things Should Look Like 3/13/2012 6

7 What They Really look Like 3/13/2012 7

8 What Keeps Us Out Of Trouble? 5+ Years of intense training after medical school Inherent skill, experience Intuition, touch, being luckier than good. Does these characteristics meet an engineers standards for reliability, reproducibility, QC? 3/13/2012 8

9 Available Technology 3/13/2012 9

10 We Need Engineering Solutions Develop an imaging system that penetrates opaque tissues Identify important structures by their unique chemical composition 3/13/

11 Solutions Infrared imaging-penetrates tissue Spectral Imaging-Identify the chemical spectrum for bile 3/13/

12 Bile is Green 3/13/

13 Hyperspectral Imaging 3/13/

14 LCTF HSI 3/13/

15 Figure 1. In vivo visible-reflectance hyperspectral imaging system. a, Broadband light source produces radiation, which is reflected off a subject and spectrally discriminated by a liquid crystal tunable filter (LCTF), transduced by a CCD detector, and digi... Copyright American Heart Association Zuzak K J et al. Circulation 2001;104:

16 Apparent Absorbance Apparent Absorbance Apparent Absorbance Near Infrared Macroscopic Hyperspectral Imaging Gallbladder, Cystic Duct and Liver Tissue Zuzak & Livingston Cystic Duct 1.0 Gallbladder Wavelength nm Liver Wavelength nm Principle Component Analysis Wavelength nm 60 pound swine, post mortem

17 Problems The system was too slow 1 minute acquisition time Liquid crystal filter was limiting Too slow for clinical use Spectral artifacts from motion 3/13/2012 SURGERY@utsouthwestern.edu 17

18 Solution Collaboration between UTSW and UTA Partner with TI and Elcan Develop a DLP based system to generate light of specific wavelengths at discrete time intervals Vast system performance improvement 3/13/2012 SURGERY@utsouthwestern.edu 18

19 DLP 3/13/

20 Initial HSI 3/13/

21 Image Frame # 26 of 101 (Wavelength = ) Apparent Absorbance (a.u.) Apparent Absorbance (a.u.) Case 07 (Schwarz) CBD Arteriole Wavelength (nm) Wavelength (nm) CBD Arteriole CBD Arteriole

22 Image Frame # 26 of 101 (Wavelength = ) Procedure notes state middle structure is CBD Case 08 (Schwarz)

23 Apparent reflectance intensity (a.u.) 1.6 Unfiltered Spectra CBD CBD CBD CBD (2) CBD CBD CBD CBD Wavelength (nm)

24 Normalized apparent reflectance (a.u.) Filtered and Normalized Spectra CBD CBD Avg norm spectrum CBD (1) CBD (2) CBD CBD CBD CBD Wavelength (nm)

25 Hemoglobin Least Square Solution Obtain pure substance spectral standards (saturated and destaurated Hgb)-Vector P Projection Matrix Project the observed spectra onto the saturated and desaturated Hgb planes to get the amount of Hgb 3/13/2012 SURGERY@utsouthwestern.edu 25

26 Apparent Absorbance A xy (λ i )= -log 10 (R xy (λ i ) / R xy (λ i ) ) Reference Spectral Components: Oxyhemoglobin and Deoxyhemoglobin Oxyhemoglobin Deoxyhemoblobin Wavelength (nm)

27 Deconvolution Example Wavelength (nm) Measured Superposition HbO 2 Reference Hb Reference

28 Deconvolution Example Wavelength (nm) Measured Superposition HbO 2 Reference Hb Reference

29 Deconvolution Example Wavelength (nm) Measured Superposition HbO 2 Reference Hb Reference

30 DLP Hyperspectral Imaging System COMPUTER Synchronized Hardware Control Image Processing Programmable Spectral Light Source DLP Inside Focal Plane Array Tissue Sample Acquire Data Cube Digitized Hyperspectral Image Cube Visualization

31 Graphical User Interface (GUI) DLP Hyperspectral Imaging System GUI developed by:

32 Timing Diagram for DLP HSI with 126shot Method Start Acquisition (n = 1) Command OL 490 to Illuminate Spectrum n n = n+1 HQ2 Open Shutter Expose CCD 40 ms t shot = ms Close Shutter t acquisition = * 126 = ms A/D conversion Save as temp00n.dat no Is n = 126? yes Process with Oxyz Algorithm in MATLAB t processing = ms Output bitmap image T total = ms

33 Timing Diagram for DLP HSI with 3shot Method Start Acquisition (n = 1) Command OL 490 to Illuminate Spectrum n n = n+1 HQ2 Open Shutter Expose CCD 1.25 ms t shot = 93.3 ms Close Shutter t acquisition = 93.3 * 3 = 280 ms A/D conversion Save as temp00n.dat no Is n = 3? yes Process with 3shot Algorithm in MATLAB t processing = 110 ms Output bitmap image T total = 390 ms

34 Relative % HbO2 Normalized to Control Video Rate Hyperspectral Imaging Biochemical Visualization of Human Kidney Over Time Relative Contribution of HbO 2 Control 8 min after clamping 1 min after clamp off Clamped Control Iced Cut Series1 Poly. (Series1) R² = Clamp Off Time after Clamping (minutes) Images from Human Partial Case 15 ( ) and Plot of all Cases

35 Animal Study DLP Hyperspectral Imaging Picture of pig kidney immediately after arterial clamping Assuming arterial blood is shunted away from kidney by arterial clamp Invasive Lycox probe inserted into kidney Kidney

36 DLP Hyperspectral Imaging Kidney Hyperspectral Image discovers Pig Kidney with 2 Renal Arteries Relative Contribution of HbO 2 -Discovery: The Kidney had two renal arteries. Only the lower pole artery was clamped, Hyperspectral Imaging immediately indicated continued perfusion of the upper Kidney pole. 10

37 DLP Hyperspectral Imaging Kidney Hyperspectral Image of Pig Kidney with Both Renal Arteries Clamped Relative Contribution of HbO 2 Re-situating the clamp over both arteries led to ischemia of both upper lower poles. 10

38 Pig Partial Nephrectomy Trial 2 Note:- 3 shot Images showing more oxygenation in intestines as kidney becomes deoxygenated 5 min 32 sec before clamping 4 min 44 seconds before clamping Clamp on 1 minute after clamping 2 minutes after clamping 3 minutes after clamping 100 Clamp On % HbO Kidney Intestine Time before and after Clamping in Minutes

39 Relative Percent HbO2 Pig Partial Nephrectomy Case 1 (Kidney and Intestines) Control 27 minutes before clamping 5 minutes after clamping 15 minutes after clamping 34 minutes after clamping 57 minutes after clamping 1 minute after clamp off 14 minutes after clamp off 120 Clamp On (100%) Clamp Off Kidney Intestines Time before and after Clamp (Minutes)

40 Relative Percent HbO2 Human Partial Nephrectomy Case 3 Control (0 min) After Clamping (3 min) Clamped & After Icing (11 min) Clamped (15 min) Clamped (21 min) Clamped (31 min) After Clamp Off (33 min) Clamp Off (1 hr 5 min) Partial Nephrectomy 100 Clamp On Clamp On & After Icing Clamp Off % HbO Time in Minutes

41 In Vivo, Visible Reflectance, Hyperspectral Imager Coupled to a Zeiss Neurosurgical Microscope Source Illumination Reflection Of Subject

42 Brain Surgery Monitoring for Preventing Post Surgery Stroke Control After Induced Ischemia % HbO2 Normal Tissue Damaged Tissue

43 DLP Hyperspectral Microscope Monitoring Diabetic Retinopathy Double Knock Out (Apoe -/-, db/db) Mouse Model

44 Relative Oxyhemoglobin Double Knock Out Diabetic Mouse: Hom Apoe/ Hom db 90 Day: 78 Day: B Parental 1: Hom Apoe/ Wildtype db 50 Day: 79 Day: Parental 2: Wildtype Apoe /Hom db Day: 81 Day: 151 Data Collected Using LCTF

45 LCTF Hyperspectral Imaging System For Imaging Human Retina Illumination Mirror CCD Beam Splitter 0 Subject s Head rest Visible LCTF Common Center of Rotation Eye Piece Magnification Knob Light Source Joystick Intensity Control Knob Table Top

46 Apparent Absorbance Relative Oxyhemoglobin Contribution Apparent Absorbance % HbO Microscopic Hyperspectral Imager: Human Retinal Imaging of Oxyhemoglobin Contribution P < Area 1 Area Wavelength (nm) Gray-scale Encoded Hyperspectral Image Wavelength (nm)

47 Hyperspectral Fundus Camera Laptop CCD Camera LLG Spectral Light Source Fundus Camera

48 Need to Improve Visualization Algorithms artifacts Relative HbO 2

49 In Vivo Hyperspectral Imaging of Human Tissue: Spatial Variation of Percentage of HbO 2 and Surface Temperature in Response to Burn Apparent Absorbance A xy (λ i ) = -log 10 (R xy (λ i ) / R xy (λ i ) ) Bright Field 1 inch Thermal Image 1 inch 1 inch Multivariate Least Squares with respect to Oxyhemoglobin Spectra Under points and Wavelength (nm)

50 BURNS Color Photo Chemically Encoded HbO 2 Chemically Encoded H 2 O Relative Contribution of HbO Relative Contribution of H 2 O

51 BURNS Post Fasciectomy Color Photo Chemically Encoded Oxyhemoglobin Chemically Encoded Water Relative Contribution of HbO Relative Contribution of H 2 O

52 Suture Tension Session Session Digital Image Zuzak & Livingston NIR HbO 2 Vis HbO 2 Patient 002, Monitoring Post Amputation Left Lower Limb, Foot, left BKA A A B 84.5±2.1 A B 62±1.2 B1 Percentage of HbO A B A 0 Percentage of HbO 2 B1 A A B 70±3.2 B2 B A B 67.6±

53 Percentage of H 2 O Percentage of H 2 O Percentage of H 2 O Monitoring Post Amputation Recovery Percentage of H 2 O Percentage of H 2 O Percentage of H 2 O A B 76.3±0.86 A B 56.2±0.63 Session I A A B B Patient 002, Amputation, Left AKA Patient 005, Amputation, Left AKA Patient 006, Amputation, Left BKA A B 81.58±4.38 A B Patient 002, 005, 006

54 Percentage of Oxyhemoglobin Percentage of oxyhemoglobin Hyperspectral Imaging Visible NIR % HbO % HbO 2 Data collected 10 th minute after removing the tight shoe

55 Diabetic Neuropathy / / / / / /

56 Other Potential Applications The speed and versatility of DLP technology is making Hyperspectral Imaging practical for a wide variety of surgical and clinical applications. Neurological Surgery - Preventing Stroke After Brain Tumors. Plastic Surgery - Skin flaps, Wound Management and Burns. Surgery - Non-Invasive Laparoscopic Optical Biopsy During. Urology Tumor Removal during Kidney Surgery. Ophthalmology Monitoring Diabetic Retinopathy. Clinical Monitoring Managing Lower Limb Amputations.

57 Versatility 3/13/

58 Fluorescence of ICG Vascular Infusion and DLP Spectral Excitation Picture of Placenta Prior to ICG infusion Placenta Control Prior to ICG infusion Vascular infusion Of ICG via cannual The DLP Illuminates tissue with predetermined spectral illumination exciting the ICG to fluoresce.

59 Major seed funding provided by Texas Instruments Supplemental funding provided in part by Hudson-Pen Endowment Smith Endowment Department of Energy.

60 BME in the Metroplex 3/13/

61 1972 3/13/

62 The Future 3/13/