VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY -------------------- LÊ NHẬT TÂN AN END-TO-END PIPELINE FOR INTRACRANIAL HEMORRHAGE BRAIN IMAGE REGISTRATION Major: Engineering Physics Major code: 8520401 MASTER’S THESIS HO CHI MINH CITY, July 2023 THIS THESIS IS COMPLETED AT HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY – VNU-HCM Supervisor 1: Assoc. Huynh Quang Linh Supervisor 2: Prof. Sozo Inoue Examiner 1: Prof. Phan Bach Thang Examiner 2: PhD.
Hoang Manh Ha This master’s thesis is defended at HCM City University of Technology, VNU- HCM City on July 23rd, 2023. Master’s Thesis Committee: 1. Ly Anh Tu 2. Pham Thi Hai Mien 3.
Phan Bach Thang 4. Hoang Manh Ha 5. Pham Tan Thi Approval of the Chairman of Master’s Thesis Committee and Dean of Faculty of Applied Science after the thesis being corrected (If any). CHAIRMAN OF DEAN OF FACULTY OF THESIS COMMITTEE APPLIED SCIENCE NATIONAL UNIVERSITY HCMC SOCIALIST REPUBLIC OF VIETNAM UNIVERSITY OF TECHNOLOGY Independence – Liberty – Happiness TASK SHEET OF THE MASTER’S THESIS Name : Lê Nhật Tân ID : 1970501 Date of Birth : February 25, 1997 Place of birth : Binh Thuan Major : Engineering Physics Major code : 8520401 I.
THESIS’S TITLE : An end-to-end pipeline for Intracranial Hemorrhage Brain Image Registration. TASKS : Constructing a comprehensive procedure that integrates intracerebral hemorrhage images and aligns pathological images with a standardized image atlas. Addressing the obstacles inherent in training images containing irregular structures within a practical deep learning model for image alignment. DATE OF ASSIGNMENT : January 2023.
DATE OF COMPLETION : June 2023. Huynh Quang Linh, Prof. Sozo Inoue Ho Chi Minh city, ………………, 2023 SUPERVISOR HEAD OF DEPARTMENT (Name & Sign) (Name & Sign) DEAN OF FACULTY OF APPLIED SCIENCE (Name & Sign) ACKNOWLEDGEMENT I would like to thank for Doctor Pham Tan Thi and Assoc. Prof Huynh Quang Linh from Ho Chi Minh University of Technology, who inspired me to follow the research career.
Your meticulous attention to detail and thorough understanding of the subject matter have constantly challenged me to think critically and strive for excellence. I would like to convey gratitude to my supervisor Professor Sozo Inoue for his strong encouragements and directions. Although this Master course was a challenge for me due to the Covid 19 spreading and my first abroad journey, my knowledge and future research direction has been continuously developing under his supervising. I would also like to thank Syoji Kobashi Sensei from University of Hyogo, Koichi Arimura Sensei from Kyushu University Hospital, Koji Iihara Sensei from National Cerebral and Cardiovascular Center Hospital for creating an interesting research on brain medical image and providing me valuable comments in my medical imaging research works.
I want to give a special thanks to Nishimura san, Uchida san, and Yoshinaga san, who helped me a lot when I first arrived in Japan; for my peers Kaneko san, Adachi san, Kazuma san, Ryu san, Nazmun, my seniors Defry, Fikry, Alia, Farina, and Tina and juniors Min chan, Quynh, Miyake san, who made my study time in Japan greater and more enjoyable. With deep appreciation, Le Nhat Tan i ABSTRACT Intracranial Hemorrhage is a dangerous intra-brain bleeding event, that threatens human life if there is no timely recognition and treatment. To assist the diagnosis and treatment planning for this serious event, attributes of the bleeding behavior should be analyzed over an extensive population of patients. Non-rigid image registration could be a effective tool to simplify this analysis process due to its ability to transfer characteristics of the abnormality from the collected data to the certain reference template.
However, current advances in image registration still face a huge challenge due to the dissimilarity in image intensity among the images of disease subjects. This thesis proposed a comprehensive pipeline including a traditional transformation and healthy tissue generation to handle the high-variance real-world collected dataset and the dissimilarity challenge in the Computed Tomography image of the Brain with Intracranial Hemorrhage, which is the new image scenario in this field of research. The performance of the proposed pipeline was compared with the state-of-the-art deep learning model including with the traditional transformation through visual inspection, hematoma change rate and sum square different metrics. The test result presented that the similar-based model performance was substantially fluctuated by the abnormal volume, while our pipeline significantly reduces its affection to improve the hematoma displacement and preservation, and keeping the considerable performance in the non-hematoma structure similarity.
The approach proposed in this work could remarkably contribute to further study on disease behavior analysis. ii THE COMMITMENT OF THE THESIS’ AUTHOR I, Le Nhat Tan, hereby declare that the master thesis titled "An end-to-end pipeline for Intracranial Hemorrhage Brain Image Registration" submitted to Ho Chi Minh University of Technology is a genuine and original work conducted by myself under the guidance of Assoc. Prof Huynh Quang Linh and Prof Sozo Inoue. Thesis author, Le Nhat Tan iii TABLE OF CONTENTS ACKNOWLEDGEMENT.
ii THE COMMITMENT OF THE THESIS’ AUTHOR. iii FIGURE LIST. vi TABLE LIST. viii ACRONYMS AND ABBREVIATIONS.
ix CHAPTER 1: INTRODUCTION .1 Research Background and Practical Significance .5 CHAPTER 2: LITERATURE REVIEW. From Traditional to Deep Learning Approaches. Similarity-based Deep Learning Approaches. Image Registration for Data with Abnormality.
End-to-end pipeline. Image Pre-alignment by Affine Transformation. Deep Similarity-Based Model .24 CHAPTER 5: RESULTS AND ANALYSIS. Non-hematoma structure mapping .51 v FIGURE LIST Figure 1.
1: The Intracranial Hemorrhage with a bleeding region occurs in the Computed Tomography (CT) and Magnetic Resonance Image (MRI). 1: The bleeding region annotation by physicians. There is slices representative for 3 groups of hematoma volume with the hematoma to brain ratio is less than 0.1 (B), and larger than 0. The marked area is represented by the green line surrounding the bleeding area………………………14 Figure 3.
2: The hematoma-brain shape volume ratio of all the test Brain ID. 1: The proposed image registration pipeline. In which the real-world origin source image experiences the inpainting process and the affine trans-formation before feeding into the deep learning model. The output of deep learning model, the deformable vector field, is used to warp the pre-align source data……………….
After inpainting process, the normal tissue within masked region are generated (right). 3: The affine transformation process. The Original image dataset (left) is pre-alignment by affine transformation with the target space as the Reference image. 4: In the encoder (gray color), the stridden convolution was used.
One more layer was added to the decoder (blue color) to transform the results into a flow field. The number in each layer represents the number of used kernels, the below number is the resolution respectively with the size of each slice after the convolution and deconvolution process. In training stage, the output DVF is used to warp the Inpainted Source image. The loss optimization process includes the MSE loss is calculated by the difference between the Warped image and the Reference image, and the regularization loss is calculated within the transformation field (the DVF).
6: SSD Calculation Process. The Reference and the Transferred slice are excluded the hematoma region before calculating. 1: Visual Inspection of small-volume hematoma. Columns from left to right are:Original slices, Reference slices, Results of DL pipeline and Results of DL-IP pipeline.
2: Visual Inspection of high-volume hematoma. Columns from left to right are: Original slices, Reference slices, Results of DL pipeline and Results of DL-IP pipeline. 3: Visual Inspection of hematoma locates near by the skull structure. Columns from left to right are: Original slices, Reference slices, Results of DL pipeline and Results of DL-IP pipeline.
4: Line plot of the hematoma change rate (excluding skull-close cases) after processing by the Deep Learning alone (orange) and with Image Inpainting (blue) follow by the hematoma volume. 5: The box plot of SSD value of DL (right) and DL-IP (left) pipeline. 6: The Sum Square Difference value of DL (orange) and DL-IP (blue) pipeline follow by the hematoma volume plot. 1: The affection of hematoma volume on the hematoma displacement performance.
2: The affection of hematoma location on the hematoma displacement performance. 3: Comparison with and without using Affine Transformation as a Pre- aligment process. Columns from left to right are: Original, Pre-aligment, Reference, the results on Original, and the results on Pre-alignemnt images.41 vii TABLE LIST Table 2. 1: Comparison of traditional standard and deep learning method.
1: Result summary of the visual inspection………………………………30 Table 5. 2: Summary of the hematoma change rate results. 3: Summary of the SSD results.33 viii ACRONYMS AND ABBREVIATIONS Acronym/Abbeviation Explaination 1 AI Artificial intelligence 2 ANTs the Advanced Normalization Tools 3 CC Cross-Correlation 4 CT Computed Tomography 5 DL Deep Learning 6 DL-IP Deep Learning and Image Inpainting Combination (The proposed method) 7 DVF Deformable Vector 8 ED Emergency Department 9 FCN Fully Convolutional Network 10 HCR Hematoma Change Rate 11 ICH Intracranial Hemorrhage 12 MI Mutual Information 13 MRI Magnetic Resonance Imaging 14 MSE Mean Squared Error ix 15 SPM Statistical Parametric Mapping 16 SSD Sum Squared Error 17 SyN Symmetric Normalization x CHAPTER 1: INTRODUCTION 1.1 Research Background and Practical Significance Intracranial Hemorrhage (ICH) is a life-threatening event that happens when a bleeding region occurs inside the human brain, leading to a high mortality rate without timely recognition. In an ICH event, the presence of a bleeding area [1, 2], or hematoma region, could be inside the brain tissue (such as Intracerebral Hemorrhage [3], Cerebral microhemorrhage [2]), or between the brain tissue and the skull (such as Epidural [4], Subdural Hemorrhage [5]).
The appearance of this unusual fluid leads to the continuous increment of the intracranial pressure [6], one of the main causes of the deadly event, strokes. The common causes of ICH are brain damage by external factors [6], called traumatic ICH, and the consequence of vessel- related diseases such as hypertension and amyloid angiopathy, called non-traumatic ICH [6]. According to the enabled population studies from 1980 to the end of 2008 summary [7], within one month after recognition, the incidence of ICH significantly increased and the mortality of 40. Therefore, fast and accurate diagnosis assistant tools are needed to provide timely treatment planning.
Moreover, summarizing hematoma characteristics in a particular population of ICH patients could remarkably contribute to disease behavior analysis. In this work, an image processing pipeline is deployed to support the abnormality's attribute summarizing on the ICH brain images dataset. CT is an anatomical imaging method that can detail provide human organ structures based on their differences in X-ray energy absorption. The image acquisition takes a few minutes to complete a brain CT scan.
This is significant shorter compared to another anatomical imaging, the Magnetic Resonance Imaging (MRI), which takes around 30 to 60 minutes for a brain scan. In the acquired image, the hematoma region can be easily recognized as 1 a high gray level area, mostly in white color, standing out against the background structures of low pixel values. Therefore, CT is the most effective method for fast ICH screening and monitoring. In this study, the collected CT scan of the brain with ICH (in Chapter 3) is used for image processing deployment.
1: The Intracranial Hemorrhage with a bleeding region occurs in the Computed Tomography (CT) and Magnetic Resonance Image (MRI). [2] Image Registration is one of the most concerning research topics in the medical image analysis field due to its wide applications. In clinical diagnosis, a medical image acquisition process could be implemented in different modalities, patients, or practice time. To productively mine the diagnosis-support information, unlike the rigid transformation which just handles the object relocation, the non-rigid image registration is commonly conducted to map all the characteristics of anatomical structures in the Source image to the matching structures in the Reference image.
Depending on the application, the Source and Reference image could be taken by the same imaging modality (unimodal) or a different modality (multimodal), in the same patients at different times (intra-patient) or in various patients (inter-patient).