Xiaopeng Yang 1, Younggeun Choi 1, Wonsup Lee 1, Dr. Ji Hyun Kim 2, Dr. Hee Chul Yu 2, Dr. Baik Hwan Cho 2, and Dr. Heecheon You 1

Transcription

1 Xiaopeng Yang 1, Younggeun Choi 1, Wonsup Lee 1, Dr. Ji Hyun Kim 2, Dr. Hee Chul Yu 2, Dr. Baik Hwan Cho 2, and Dr. Heecheon You 1 1 Department of Industrial and Management Engineering, Pohang University of Science and Technology, South Korea 2 Department of Surgery, Chonbuk National University Medical School, South Korea 1

2 Members IME Department Biakhwan Cho Heechul Yu Heechon You Xiaopeng Yang Younggeun Choi Wonsup Lee Jihyun Kim 2

3 Contents 3

4 Virtual Liver Surgery Planning System? A system which assists surgeons in preoperative planning for liver resection and transplantation Image Processing 3D Reconstruction of the liver, vessels, and tumor(s) from CT volume dataset Liver segmentation Location & size of the tumor(s) Vascular structure Liver segments Resection plane Anatomical Visualization Safe & Rational Surgery Quantitative Analysis Volumetry of the liver Volumetry of the remnant and/or graft TFLV = 1237 ml Remnant = 362 ml %RLV = 30% 4

5 Generic Virtual Surgery Systems Rapidia (Infinitt) Voxar 3D (Barco Toshiba) Syngo.via (Simense) OsiriX (Freeware - Pixmeo) System Country System features Korea Japan Germany Swiss 3D visualization Measurement Options Colon & polyps Cardiac Ca Vessel Brain angio 3D visualization Measurement Options Colon Cardiac Ca Vessel Oncology PET & CT segmentation (extraction) CT Lung CAD Colonography Cardiology Vascular Cardiac Ca Scoring Neurology PET evaluation Perfusion CT 3D visualization Measurement Options Custom plugins iphone, ipad compatible 5

6 Generic Virtual Surgery Systems: Limitations Lack of functions specialized for liver surgery planning, such as liver segmentation and surgical resection simulation Liver extraction Manual drawing Liver contour manually traced slice by slice Time demanding: more than 30 min for a CT dataset of 200 slices with a thickness of 1 mm Semi-automatic Simple region growing method provided Heavy manual editing required due to false extraction Cumbersome to users 6 Voxar 3D Syngo.via Rapidia OsiriX

7 Specialized Virtual Liver Surgery Planning Systems System Visia Synapse Vincent (MeVis) (Fujifilm) MeVis Distant Service Synapse Vincent Visia Visia Synapse Vincent (MeVis) (Fujifilm) (MeVis) (MeVis) Synapse Vincent (Fujifilm) (Fujifilm) IQQA Liver (EDDA) IQQA Liver IQQA Liver (EDDA) IQQA Liver (EDDA) (EDDA) Mint Liver (Mint Medical && German Mint Liver Cancer (Mint Research Medical Center) & German Cancer Research Center) Mint Mint Liver Liver (Mint (Mint Medical Medical & & German German Cancer Cancer Research Research Center) Center) Country Country Country Country Germany Germany Japan Japan Germany Germany Segmentation of the Japan Japan Segmentation of the Liver extraction Liver extraction liver, veins, biliary Vessel analysis Liver segmentation Segmentation liver, veins, of biliary the Liver Vessel analysis Segmentation of of the the Liver extraction system, system, and tumors and tumors Liver segmentation liver, veins, liver, biliary veins, Volumetric Volumetric biliary data for data Vessel for Vessel Volumetry analysis Volumetry analysis system, system, and and both tumors and remnant both remnant tumors and/or and/or Liver Segmentation segmentation Surgery planning Volumetric System graft graft System Volumetric data for for data for Volumetry features Vascular Vascular territory territory Volumetry both features remnant and/or Surgery planning System both remnant evaluation evaluation Virtual and/or resection Surgery planning graft Virtual resection System features graft options options Vascular territory features evaluation Vascular territory Virtual evaluation resection options Virtual resection options Surgery planning USA USA China China Germany Germany USA Germany USA Segmentation China of the Segmentation of the Integration Integration in the in the liver, lobes, vessels, existing software liver, lobes, vessels, existing software Segmentation of of the the Integration in in the the and lesion and lesion platform platform liver, Volumetry lobes, vessels, Automatic Automatic liver existing liver software Calculation of 3D and and lesion analysis analysis platform distances, distances, margins, margins, Visualization Visualization Volumetry Automatic liver liver and diameters and diameters Volumetry Volumetry Calculation Left and right of liver of 3D 3D Calculation of 3D Definition Definition analysis of analysis resection of resection volume volume distances, and and margins, strategies strategies Visualization distances, percentage calculation percentage calculation margins, Visualization and and Virtual diameters knife control Volumetry Left and Left Collaboration and diameters and right right with liver liver Definition Volumetry of of resection volume Virtual colleagues and knife online colleagues online and Definition strategies of percentage control calculation resection strategies Virtual knife control Collaboration with with colleagues online Volumetry Calculation of 3D Left and right liver Virtual knife control Collaboration with 7

8 MeVis Distant Services Provided by MeVis Medical Solutions AG, Germany LiverAnalyzer & LiverViewer developed using MeVisLab Analysis services Segmentation of the liver, veins, biliary system, and tumors Volumetric data for both remnant and/or graft Vascular territory evaluation Virtual resection options Limitation: LiverAnalyzer not for sale 8

9 Surgery Planning Oncological Case Liver analysis report (viewed by LiverViewer) Limitation: Difficult to crosscheck the accuracy of the analysis results since CT images are not provided 9

10 Synapse Vincent Developed by Fujifilm, Japan Functions provided Liver extraction Vessel analysis Liver segmentation Volumetry Surgery planning Limitations: Expensive More user-friendly UI needed Manual drawing of vessels required Cumbersome to users 10

11 Research Objectives Development and Usability Testing of Dr. Liver, a User-Centered Virtual Liver Surgery Planning System 1. Functions specialized to liver surgery Surgery planning & Volumetry 2. Decision support information 3. Intuitive, user-friendly interface 3D Digital Liver Extraction User Segmentation 4. Acceptable processing time Vessel Extraction Tumor Extraction 11

12 Use Scenario Development A use scenario consisting of hierarchical tasks was developed based on Interviews with surgeons Benchmarking of commercialized systems Literature survey Questionnaires Liver extraction Vessel extraction Tumor diagnosis Liver segmentation Surgery planning 3 ~ 5 min 8 ~ 10 min 3 ~ 4 min 7 ~ 10 min 2 ~ 3 min S1. Seed point selection S2. Liver extraction S3. Liver editing S4. Update & save S1. Mask the liver S2. Seed point selection S3. Vessel extraction S4. Vessel editing S5. Update & save S1. Seed point selection S2. Tumor extraction S3. Tumor Editing S4. Update & save S1. Segmentation plane generation S2. Liver segment adjustment Entire processing time: 20 ~ 30 min 12 S1. File loading S2. Resection of tumor by a sphere S3. Volumetry

13 Use Scenario: Liver Extraction Module S1(M). Seed point selection S2(A). Liver extraction S3(M). Liver Contour editing S4(A). Update & save Total 3 ~ 5 min < 1 min 1 min 1 min < 1 min Multiple seed points selection on the liver using the mouse Liver extraction using the proposed hybrid method Liver contour verification Liver contour editing using a scalable editing circle Update & save of the 3D liver 13

14 User-Friendly UI Features (1/4) A hierarchical and sequential user interface was designed based on the use scenario High level UI Low level UI 14

15 User-Friendly UI Features (2/4) Procedure status indication and color coding Iterative execution arrow Procedure status Indication coding Iterative execution arrow 3D view indication box Procedure status color coding 3D view resetting buttons 15

16 User-Friendly UI Features (3/4) 3D view indication and resetting functions for easier 3D object manipulation 3D view indication box 3D view resetting buttons Hot key menus 16

17 User-Friendly UI Features (4/4) Hot key menus on the 2D screen for easier accomplishment of various tasks such as seed point selection and CT image zooming in/out view resetting ons Hot key menus 17

18 Demo: Liver Extraction Module 18

19 Demo: Liver Surgery Planning Sphere Mode 19

20 Usability Testing Participants 3 medical doctors at Chonbuk National University Medical School Age: 30 ~ 50 years Patient dataset One dataset of abdominal CT images provided by Chonbuk National University Medical School, South Korea Resolution: Thickness: 1 mm 20

21 Test Design 1. Pre-Test Session (1 h) 2. Test Session (1.5 h) 3. Post-Test Session (10 min) 1. Informed consent 2. Introduction of usability testing 3. Training of Dr. Liver Liver extraction Vessel extraction Portal vein Hepatic artery Hepatic vein IVC Tumor extraction Liver segmentation Plane Sphere Liver surgery planning Plane Segment Sphere Debriefing 21

22 Measure/Instrument Matrix (selected) Measures Liver Extraction Vessel Extraction Surgery Planning Instrument/ Scale Performance Accuracy (Similarity index, False positive error, and false negative error) Completion Time m m m m Comparison to golden standard Programming Number of mouse clicks m m m Programming Number of keystrokes m m m Programming Usefulness m m m Questionnaire Ease of Use m m Learnability m m m m with 7-point Likert Scales Preference Informativeness m m m Clarity m m m Tolerance m m m Satisfaction m m m 22

23 Assessment Questions: Liver Extraction (illustrated) Questions How useful is it for extracting the liver from DICOM images? How easy is it to use? How easy is it to learn the steps of liver extraction? How adequate is the information provided? How clear are the step names? How adequate is the tolerance to allow you make mistakes? Very Poor Poor Slightly Poor Fair Slightly Good Good Very Good j k l m n j k l m n j k l m n j k l m n j k l m n j k l m n 23

24 Quantitative Assessment Results (selected) Measures Liver Vessel Surgery Extraction Extraction Planning SI (%) 97.0 (0.3) Accuracy FPE (%) 2.0 (0.2) FNE (%) 2.3 (0.4) Time (min) Interaction Auto processing 1.8 (0.5) 1.3 (0.2) 0.6 (0.1) 1.6 (0.3) 1.0 (0.2) 1.2 (0.4) Number of mouse clicks Before editing Editing 30 (8) 56 (10) 20 (5) 0 (0) 10 (7) 4 (2) Number of keystrokes Before editing Editing 0 (0) 17 (10) 24 6 (4) 0 (0) 9 (3) 3 (2)

25 Qualitative Assessment Results - Overall Module Average S.D. Liver extraction Vessel extraction Portal vein Hepatic artery Hepatic vein IVC Low score: Incomplete extraction of HA Tumor extraction Liver segmentation Liver surgery planning Plane Sphere Plane Segment Sphere Large variation: high scores provided by young doctors, but a low score by a senior doctor

26 Discussion (1/3) Use scenario development based on interviews with surgeons, benchmarking of commercialized systems, literature survey, and questionnaires User-centered Clinically practical Liver extraction Vessel extraction Tumor diagnosis Liver segmentation Surgery planning 3 ~ 5 min 8 ~ 10 min 3 ~ 4 min 7 ~ 10 min 2 ~ 3 min S1. Seed point selection S1. Mask the liver S1. Seed point selection S1. Segmentation plane S1. File loading S2. Liver extraction S2. Seed point selection S2. Tumor extraction generation S2. Resection of tumor S3. Liver editing S3. Vessel extraction S3. Tumor Editing S2. Liver segment by a sphere S4. Update & save S4. Vessel editing S4. Update & save adjustment S3. Volumetry S5. Update & save Entire processing time: 20 ~ 30 min 26

27 Discussion (2/3) User-friendly UI design Procedure-based and intuitive Procedure status indication and color coding 3D view indication box and resetting buttons for easier 3D object manipulation Hot key menus on the screen to decrease cognitive workload Procedure status Indication coding Iterative execution arrow 3D view indication box Procedure status color coding 3D view resetting buttons Hot key menus 27

28 Discussion (3/3) Usability testing Improvement needed: Connection function for discontinued extraction of HA branches Usability testing at multiple centers 28

29 Liver Anatomy POSTECH 29

30 30

31 31

32 Q & A Thank you for your attention! 32

33 IQQA Liver Developed by EDDA, USA Limitations: Expensive and no specifications of the system provided No usability testing of the system provided Not available for evaluation and close comparison 33