Transcription

1 OntoDiagram: Automatic Diagram Generation for Congenital Heart Defects in Pediatric Cardiology Kartik Vishwanath, B.S. 1, Venkatesh Viswanath, B.S. 1, William Drake, M.D. 2, Yugyung Lee, Ph.D. 1 1 School of Computing & Engineering, University of Missouri-Kansas City, Missouri, MO 64110, USA 2 Section of Cardiology, The Children s Mercy Hospitals and Clinics, Missouri, MO 64108, USA In pediatric cardiology as well as many other medical specialties, the accurate portrayal of a large volume of patient information is crucial to providing good patient care. Our research aims at utilizing clinical and spatial ontologies representing the human heart, to automatically generate a Mullinslike diagram [6] based on a patient's information in the cardiology databases. Our ontology allows an intuitive way of modeling congenital defects with the structure of the human heart. A prototype system has been implemented using Protégé 1 and the Java Advanced Imaging package 2, and is currently under testing at the Children s Mercy Hospitals and Clinics. A pilot study showed that the diagrammatical representation and characterization of congenital heart defects can be used to facilitate a better understanding of a complex congenital heart defects. INTRODUCTION The art of medical practice involves the collection, assessment, distribution and understanding of an ever-increasing amount of data. In most instances, clinicians find the diagrammatic representation of anatomic information more helpful than a listing or series of statements of such findings, as visual representation is intuitively easier to understand than text information. This is especially true when depicting congenital heart disease. Dr. Charles Mullins [6] has developed a series of diagrams that have been widely used in pediatric cardiology, enabling clinicians to quickly understand complex cardiac anatomy at a glance. Mullins diagrams are most often used in the pediatric cardiac catheterization lab. The diagrams are populated and annotated with information including but not necessarily limited to oxygen saturation, pressures, size parameters, flow data (including shunts and resistances), as well as procedural information (operational instructions and guidelines). Despite the complexity of the anatomic information contained in complete series of published Mullins diagrams, pediatric cardiologists are still frequently required to modify or change the diagrams by hand. This involves either free-hand modification of a printed-out copy of a Mullins diagram, or online modification of a Mullins diagram either using a computer based graphics routine or a commercially available product such as PedCath 3. Some computerized procedural reporting packages available for either pediatric cardiac catheterization or pediatric cardiac echocardiography reporting make use of extensive databases characterizing cardiac anatomy. An initial pediatric cardiac echocardiogram, in particular, should strive to characterize the structure of the heart in a complete a manner as possible. Traditionally, Mullins diagrams are not used in association with a pediatric cardiac echocardiography report, often because the routine generation of a Mullins diagram is time intensive and laborious; a pediatric cardiologist may be called upon to read dozens of echocardiograms a day at a large pediatric center. Despite the difficulties in generating routine Mullins diagrams for echocardiogram reports, given the potential complexity of congenital heart disease, it would be useful to link the comprehensive database associated with pediatric cardiac echocardiography and cardiac catheterization computerized procedural reporting packages. Thus, we have started to develop a database driven diagram generator, which will allow the autonomous generation of diagrams depicting even complex congenital cardiac anatomy. The components of the diagram would need to be related to each other in specialized spatial ways ( in front of to the right of above etc.) and would need to connect to each other at particular points. The resulting diagram should look very similar to a commonly recognized Mullins diagram. Ideally, the diagram should be populated with anatomic and functional data obtained at the time of the procedure. In the case of an echocardiogram, this information might include size parameters, flow velocities, peak and mean gradients, and such calculated parameters such as valve areas, ejection fraction, and flow volumes. In the case of a cardiac catheterization, parameters would include oxygen saturation, pressures, size parameters, flow data (shunts, resistances), as well as procedural information as noted above. Our research called OntoDiagram (Ontology based Diagram Generation) aims at utilizing clinical and spatial ontologies representing the human heart, to automatically generate a Mullins-like diagram 3

2 based on a patient's information in the cardiology databases. The ontology we generated for this project 4 promotes an intuitive way of modeling congenital defects in the structure of the human heart. The important features of our approach are (1) hierarchical composition of components for efficient computing and maximum utilization; (2) each component is modeled in terms of spatial relationships with neighboring components, and a set of interfaces. This allows reducing the set of anomalies to a set of components and their interface points; (3) dynamic image generation with the domain descriptions of the defects mapped to image specific information of the various components. RELATED WORK The Imaging and Human Brain project [3] is one of successful FMA applications that uses ontological representations for anatomy in depiction, retrieval and manipulation of anatomical structures. The FMA model [7] provides a basis for our work, however we have extended this model to include two-dimensional diagrammatic representation of congenital heart defects. MIAKT [2] generates a medical report using a medical ontology with lexicon including image annotations. Mimos [1] describes the semantics of medical image processing but it does not allow generating a diagrammatic representation of domain due to the lack of the capacity in describing structural aspects of the image. Khan et al. [5] automatically construct an image ontology from an input image while the OntoDiagram generates images using ontologies. Cassotta et al. [4] generate images based on domain ontologies to which the image is mapped, yet the mapping schema is not described. METHODS The OntoDiagram models the structural aspects of a diagrammatic description using domain and spatial terms. The domain expert can describe the anatomical condition using his/her own domain terminology as well as using spatial terms such as to the left of, below. This domain description is finally converted to a diagrammatic representation. Heart Defect Modeling: The defect classification includes the unique spatial aspects of each defect, with respect to its deviation from the structure of the normal heart. For example, septal defects can be classified as defects that have a missing heart component (i.e., missing piece of the septum). Each defect description is modeled as an instance of the ontology. Defective portions of the heart can be modeled as image transformations (scaling, rotation, 4 etc.) of the respective normal heart equivalent. The defect descriptions are linked to the heart structure they represent by use of properties; for instance a property hasparts links each defect to a list of component instances that constitute the defect. This classification is an intuitive way to describe the defects and it also encourages reuse of existing heart components in composition of an image. The following defects types are defined as of now: 1) Abnormal growth: Heart defects which have an abnormal or new growth. For example, patent ductus arteriosis, where the patent ductus is the abnormal growth. 2) Missing parts: Heart defects that have a missing component (e.g. septal defects where a part of the septal wall is missing). 3) Transposition: Heart defects that reflect a deviation in position of heart components with respect to their position in the normal heart (e.g., Transposition of the Great Arteries). 4) Transformation: Defects that can be expressed as a change in shape of a heart component (e.g., Coarctation of the Aorta where the Aorta is narrowed). 5) Combination: These refer to the defects that can be modeled as a combination of two or more defects of types discussed above. (e.g., Transposition of the Great Arteries with VSD or Tricuspid Valve Atresia with a hypoplastic RV, VSD and Pulmonary Atresia). Heart Component Modeling: A hierarchical representation of concepts reflects the physical structure of the human heart. The heart structure can be modeled with several sub-components of the heart. This classification of heart structure is congruous with the domain perspective of heart image. The domain (ontology) makes a hierarchical classification of heart with several levels of abstraction. We have modeled 74 heart components inspired by the FMA 5 model of human anatomy. The spatial orientation of each heart component is expressed as a set of properties attached to component instances. These properties represent directions for a component in a 2D plane. In addition, they also contain information regarding overlap between components. Examples of such properties include toleftof, torightof, above, below, etc. Mapping Domain to Image: The component model generates image descriptions of components relative to their surrounding components. These descriptions are mapped to actual images corresponding to the components. The relative spatial orientation is converted to absolute image coordinates so that the image components can be assembled. The mapping between domain and image models is defined below. 5

3 The component model is a set of component descriptions of the spatial relations with components and their neighboring components and their presence in a diagram. Formally, it is represented as a six-tuple {C t, C n, D, A, O, P} where C t and C n are the target component and its neighboring component, D specifies the spatial orientation of C t with respect to C n, as {right, left, top, bottom}, A specifies whether C t and C n are attached or not, O indicate whether C t is overlapped with C n as {above, below}, P specifies whether C t is present or not. For example, {Descending Aorta, Aortic Arch, bottom, attached, below, present} means that Descending Aorta is located bottom to the Aortic Arch and they are attached, Descending Aorta and Aortic Arch are overlapped and Descending Aorta is below Aortic Arch and Descending Aorta is present in the diagram. Pulmonary valves, Aorta and Aorta valves, etc.) (2) the relationships between the heart base components and the second layer components. In a Mullins-like diagram (Figure 1), the base of the heart (layer 1) is shown highlighted and the dotted lines represent other parts of the heart (layer 2). The relationships between components are represented as a set of interfaces. Each interface is associated with a set of assembly points that are either interface points or gate points. The interface points are defined between components, such as the points between ascending aorta and the aortic arch. The gate points are defined between components in the second layer and components in the first layer such as the points between connecting ascending aorta and septal wall of heart. The structure model is represented as a four tuples {H b, H s, G p, I p }, where H b is a set of the heart base components {C b1, C b2,,c bn } and H s is a set of other components {C s1, C s2,, C sn }, G p is a set of gate points to describe the relations between the heart base and other components {G 1 <C b1, C s1 >, G 2 <C b2, C s2 >,, G n <C bn, C sn >}, I p is a set of interface points describing the relations between components {I 1 <C i1, C j1 >, I 2 <C i2, C j2 >,, I n <C in, C jn >}. Figure 1. Base of the heart and gate points The conversion model describes how a component is transformed into a different component using some important transformations such as rotation, scaling, shearing, and perspective transformations. The conversion model is represented as a five-tuple {C, T, R, X, Y}, where C specifies the component for which the transformation rules are applied, T specifies a transformation type such as rotation and scaling, R specifies the angle of rotation, X and Y specify scaling by factor X (for x axis) and factor Y (for y axis), respectively. For example, {Ascending Aorta, rotation, 55, 0, 0} means Ascending Aorta is transformed by rotating the axes at angle of positive 55 degrees. The structure model defines (1) the heart is considered to be made of two layers: the first layer consisting of the chambers of the heart (wall of atria and ventricles), which we arbitrarily have called the heart base, and the second layer consists of the other components of heart (Pulmonary artery and Figure 2. Interface points Figures 1 and 2 show the gate points defined between pulmonary artery and septal component of heart and the interface points between ascending aorta and aortic arch, respectively. The reason for the two layered structural modeling is to provide some flexibility and scalability in component design and composition. The heart base plays a key role as each component s position can be configured respective to the heart base, making the composition process more efficient and effective. This model facilitates component level composition allowing reduction of specific cardiac anomalies to a set of components and their interface points. The image model specifies image specific information such as annotation for defect description and lab measurement, and composition rules for special image patterns such as operational marks.

4 Each defect description can be associated a set of annotations. The annotation hierarchy is driven by the physiology, for example, pressures at various point of the heart are classified as pressure_measurements, blood velocities are classified under velocity measurements. This approach enables addition/removal of information layers to a diagrammatic representation based on user preferences or defect requirements. Each heart component is typically linked to a set of annotation instances that can be used to best describe its physiology, through a property hasannotation. Transformation and Mapping: Our image generation process is a three-step process: (1) the composition of the heart base components (2) the composition of the components of the secondary layer (3) the composition between the secondary layer components and the base layer components. The interface points of component are defined as x and y coordinates of 2D component image stored in separate bitmap files. Because these components are stored this way, the coordinate system across the bitmap images may not be uniform. When source and target components are composed, they may not have the same reference coordinate system. Before compiling the image components some transformations are required. For this purpose, we modeled the composite operation, perspective transformations and geometric transformations such as rotation, scaling, and shearing. Given the two metrics of the target and source components, M t [x, y] and M s [x, y] and the interface point of target and source components [x t, y t ] and [x s, y s ], the transformation operations are described below: Composite (M t [x, y], M s [x, y]) that composes M s [x, y] into M t [x, y]. Translate ([x t, y t ], [x s, y s ], M s [x, y]) that translates M s [x, y] according to the perspective transformations from [x t,, y t ] to [x s, y s ] Rotate (M s [x, y], δ) rotates it by the angle δ Scale (M s [x, y], scale_x, scale_y) that scales M s [x, y] by scale_x and scale_y For example, if a component is rotate by δ and scaled by scale_x and scale_y and is composed to a target component, the effective changes of the source component are computed as follow: M t [x, y] = Composite (M t [x, y], Translate ([x t, y t ], [x s, y s ], Scale (Rotate (M s [x, y], δ), S x, S y )). Component Composition using Color Scheme: The composition is done (1) hierarchically, starting with fine-grained component composition followed by coarse-grained component composition (Figure 3) (2) incrementally by combining two components at a time through the steps necessary for a thorough composition of the components. At each step the images are transformed or composed using the operations discussed earlier. The two layered heart structure model utilizes a layered approach that achieves a high degree of flexibility in terms of support for the component composition. In the component composition, there might be some complex situations where components are interwoven; subcomponents or parts of the components are overlaps above or below themselves. In Figure 3, the three parts of heart (pulmonary veins, ascending aorta and left atrial wall) are interwoven: some part of the aorta overlap above the pulmonary artery and below pulmonary artery. (g) (a) (i) (b) (h) Figure 3. Composition using Color scheme To handle this complex case, we developed the following steps: (1) utilize the hierarchical component model to identify an appropriate set of components and describe the spatial relations (below, above) (2) determine the order of components based on the component description. We use a color-based component ranking schema. Each component will be assigned with only one color (an RGB value) that represents its rank in the composition. A higher ranked component appears in front of a lower ranked component should they be in the same layer. Thus, some components are visible while others are not. The step-by-step diagram generation using the color scheme is shown in Figure 3. The ascending aorta is shown below the main pulmonary artery while the left pulmonary artery is shown above the descending aorta. Once the assembly is complete, the final output generated is de-colored. The color schema facilitates determination of the rank of components during assembly versus assigning an arbitrary rank to the components. (j) (c) (d) (e) (f)

5 RESULTS The domain model contains 106 concepts, 31 properties and 135 instances. The domain model ontology was created using the protégé 1 in OWL 6. The query tool was implemented in Java using the Jena 7.The image generation module was implemented using the Java Advanced Imaging package 2. Following is the intuitive query interface that allows physicians to query heart defect descriptions and generate diagrams based on image features. data including oxygen saturation, pressures, size parameters, flow data, as well as procedural information. Figure 4(bottom) shows a diagram generated for Transposition of the Great Arteries (TGA) with Ventricular Septal Defect (VSD). This is an instance of a combination of missing parts (VSD) and transposition (TGA): The Aorta arises from the right ventricle and receives deoxygenated blood, whilst the Pulmonary Artery arises from the left ventricle. In addition there is a VSD. Oxygenated blood in the left side of the heart may reach the aorta through the VSD or via the ductus arteriosus or a foramen ovale. The annotated information populated from database is shown. We are currently implementing to present the Mullins-like diagrams with echocardiography of heart through the query interface. CONCLUSION We have introduced a hierarchical modeling and composition process for the database driven generation of a diagram depicting various congenital heart defects. The domain descriptions of the congenital heart defects are mapped using the ontology to generate a diagrammatic representation of defects. This approach has the additional benefits of selecting an appropriate set of components, and maximizing the utility of existing models by reducing the set of anomalies to a set of components and their interface points. A prototype system is currently under testing at the Children s Mercy Hospitals and Clinics for helping to facilitate existing treatments for managing patients with severe heart disorders. Figure 4: OntoDiagram Query Interface Figure 4 (top) shows a diagram generated for patent ductus arteriosis where the ductus is an abnormal growth. Failure of the ductus arteriosus to close in the early weeks of life, as normally occurs, results in a patent ductus arteriosus (PDA), which allows blood to flow between the aorta and the pulmonary artery, leading to an increase in blood flow to the lungs. If the PDA is large the pressure in the lungs may also be abnormally elevated. The letters on the diagram denote cardiac catheterization References 1. F. Aubry, A. Todd-Pokropeka, Mimos: A Description Framework for Exchanging Medical Image Processing Results, MEDINFO pp K. Bontcheva, Y. Wilks, Automatic Report Generation from Ontologies: The MIAKT Approach. NLDB 2004, pp J.F. Brinkley, C. Rosse, 'Imaging Informatics and the Human Brain Project: the Role of Structure, Review' Yearbook of Medical Informatics, pp , M. Cassotta, B. Carolis, F. Rosis, Chiara Andreoli, M. Cicco: User-Adapted Image Descriptions from Annotated Knowledge Sources. AI*IA2001, L. Khan and L. Wang, Automatic Ontology Derivation Using Clustering for Image Classification, Proc. Of 8th International Workshop on Multimedia, D. Mayer and C. Mullins, Congenital Heart Disease, A Diagrammatic Atlas. New York: Alan R. Liss, Inc, p. 5. This material is used by permission of Wiley-Liss, a subsidiary of John Wiley & Sons, Inc. 7. C. Rosse and J. Mejino, A reference ontology for biomedical informatics: the foundational model of anatomy Source Journal of Biomedical Informatics archive Volume 36(6), 2003, pp