HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY MASTER THESIS Research on application of deep learning approach in Multiclass Segmentation for Medical Images TRINH MINH NHAT nhat.vn Control Engineering and Automation Supervisor: Assoc. Van-Truong Pham Advisor’ signature School: School of Electrical and Electronic Engineering HA NOI, 9/2023 ACKNOWLEDGEMENTS I would like to take a moment to express my gratitude to the individuals who have been instrumental in shaping the trajectory of my academic journey and the com- pletion of this report. Van-Truong Pham, my mentor and guide, has been a constant source of inspiration. His dedication to my growth as a researcher and his insightful guidance have been invaluable.
His willingness to share his ex- pertise and time has been instrumental in helping me navigate through challenges and overcome obstacles. I also want to extend my heartfelt appreciation to Assoc. Thi-Thao Tran, whose contributions have left a lasting impact on my work. Though not my primary advisor, her constructive feedback, suggestions, and discussions have added depth and perspective to my research.
Her commitment to fostering a culture of learning and exploration has been immensely beneficial. In the broader context, I am deeply thankful for the unwavering support of my family. Their belief in my capabilities and their encouragement during both the highs and lows have been my driving force. Their sacrifices and unwavering faith have kept me motivated, and I am indebted to them for their constant presence in my life.
Lastly, I would like to acknowledge the academic community, my peers, and fellow researchers who have provided valuable insights and diverse perspectives that have enriched my work. The exchange of ideas and collaborative discussions have shaped my understanding and contributed to the depth of this report. As I reflect upon this journey, I am filled with gratitude for the people who have contributed to this report’s fruition. Their collective efforts have not only facilitated the completion of this project but have also nurtured my growth as a learner and researcher.
ABSTRACT In recent times, the prominence of deep learning-based techniques for medical im- age segmentation has surged. These approaches primarily revolve around innovat- ing architectural designs and refining loss functions. Conventional loss functions in this context often rely on global measures, such as Cross-Entropy and Dice Loss, or overall image intensity, yet they may fall short in addressing complexities like oc- clusion and intensity variations. In response, this study introduces an original loss function, melding both local and global image features, reformulated within the Mumford-Shah framework.
This novel approach is extended to the domain of mul- ticlass segmentation. The proposed deep convolutional neural network leverages this new loss function to facilitate end-to-end training while concurrently achiev- ing multi-class segmentation. Furthermore, motivated by the PiDiNet architecture, I propose a new Attention-PiDi-UNet architecture. This augmentation empowers the model to fuse contextual information across dense layers, efficiently capture se- mantic insights, and avert overfitting, resulting in precise segmentation outcomes.
The proposed approach is rigorously assessed across four distinct biomedical seg- mentation datasets encompassing various imaging modalities, spanning 2D to 3D dimensions, including dermoscopy, cardiac magnetic resonance, and brain mag- netic resonance. Evaluation results on datasets like Lesion Boundary Segmenta- tion, the dermoscopic dataset, automated cardiac diagnosis, and 6-month infant brain MRI Segmentation corroborate the algorithm’s superior performance com- pared to existing state-of-the-art methods. This robustly underscores the potency of our multiclass segmentation approach for diverse biomedical images. Student’s signature TABLE OF CONTENTS LIST OF FIGURES ii LIST OF TABLES iii LIST OF SYMBOLS iv CHAPTER 1.1 Motivation for Participating in Medical Image Segmentation Chal- lenges .2 Advancements in Medical Image Segmentation and Innovative Ap- proaches .1 Artificial Neural Networks .2 Deep neural network .1 Glorot and He Initialization .2 Non-Saturating Activation Functions .3 Convolution Neural Network .1 The Architecture of the Visual Cortex .4 Transposed Convolutional Layers in semantic segmentation 24 2.4 U-Net based architectures .2 Compact Dilation Convolution-based Module (CDCM) .1 Mumford-Shah Loss .2 Active Contour Method.
THE PROPOSED APPROACH 31 3.1 The ISIC-2018 Dataset .2 The PH2 Dataset .3 Automated Cardiac Diagnosis Challenge (ACDC) .4 6-month Infant Brain MRI Segmentation (iSeg) Dataset .3 Active Focal Loss Function .1 Dice similarity coefficient (DSC) .2 Intersection-over-Union .3 Recall, Precision, Sensitivity and Specificity .4 Modified Hausdorff distance [79] .5 Average surface distance [79] .2 Results on ISIC-2018 Dataset .3 Results on PH2 Dataset .4 Results on ACDC Dataset .5 Results on iSeg-2017 Dataset .6 Performance of the Proposed Loss. SUMMARY 51 LIST OF PUBLICATIONS 53 REFERENCES 55 LIST OF FIGURES Figure 2.1 Threshold logic unit: an artificial neuron applies a step function after calculating the weighted sum of its inputs [39] .2 Perceptron architecture of two neurons input, one neuron bias, and three neurons in output [39] .3 Multilayer Perceptron architecture has two inputs, four neurons in one hidden layer and three neurons in output layer [39] .4 Hidden layers in Deep Neural Network [41] .5 Logistic activation function saturation [39] .9 With the stoppage regularization a random set of all the neurons is "dropped out" in each training iteration in one or more layers, with the exception of the output layer [39] .10 The visual signal progresses through the brain, neurons respond to more complex patterns in larger receptive fields [39] .11 Square local receptive fields in CNN layers [39] .12 Relations between layers and zero padding [39] .13 Reducing dimensionality the input feature map using a stride with step of 2 [39] .14 Two different filters are being applied to get other two feature maps [39] .15 Three color channels images and convolutional layers with many features’ maps [39] .16 Max pooling layer with 2×2 pooling kernel, no padding and step size stride equal 2) [39] .17 Invariance to small translations [39] .18 Example of semantic segmentation [39] .19 Upsampling utilizing a Transposed Convolutional Layer [39] .20 Spatial resolution from lower layers is recovered by Skip layers [39] 26 i Figure 2.1 My proposed network architecture Attention-PiDi-UNet.1 The representative segmentation results of my method on different skin lesions size from my testing set in the ISIC-2018 dataset.2 Representative results in PH2 dataset .3 The representative result of the right ventricle (yellow), myocardium (green), and left ventricle (blue) of three examples using my method on the ACDC 2017 challenge.4 The representative result on various slices of testing sample IDs 11, 16, and 17, respectively. The T1 weighted, the T2 weighted, and my segmentation result are indicated from left to right, respectively.5 The learning curves by the proposed method when training im- ages from four databases in terms of average DSC of classes. (a) The ISIC-2018 dataset.
(b) The PH2 dataset (c) The ACDC dataset. (d) The iSeg-2017 challenge. 49 ii LIST OF TABLES Table 4.1 Comparison with other popular approaches on the ISIC 2018 dataset. Results have been taken from [80] except for the last four methods.2 Comparison with other popular approaches on the PH2 dataset.
Results have been taken from [81] except for the last four methods.3 Comparison with other popular approaches on the ACDC dataset. DSCs on RV, Myo, LV and the average DSC have been calculated. Re- sults have been taken from [82] except for the last four methods.4 The DSC, MHD, ASD, and average metrics of segmented classes in validation dataset of 4 out of top 8 teams in [5] of the iSeg-2017 challenge and my proposed approach (MHD: mm, ASD: mm).5 Comparison with other loss function in DSC on three the datasets. 48 iii LIST OF SYMBOLS Symbols Meaning AI Artificial Intelligence CNN Convolutional Neural Networks FCN Fully Convolutional Networks CDCM Compact Dilation Convolution-based Module CSAM Compact Spatial Attention Module CE Cross-Entropy MSE Mean Squared Error DSC Dice Similarity Coefficient IoU Intersection-over-Union ASD Average Surface Distance MHD Modified Hausdorff Distance iSeg 6-month Infant Brain MRI Segmentation ACDC Automated Cardiac Diagnosis Challenge RV Right Ventricle LV Left Ventricle Myo Myocardium µB The mean of vector input, assessed over the entire mini-batch B.
σB Inertial moment around the yaw axis lf The standard deviation of the vector of input mB The plenty of cases in the mini-batch x(i) The normalized inputs for case i ⊗ Element-wise multiplication. β The output shift (offset) parameter vector for the layer ε Small number which prevents zero division (commonly 10−7 ) z(i) The Batch Normalization output operation. Ei Output of ith encoder block Di Output of ith decoder block Hi Height of ith output feature map Wi Width of ith output feature map αi Additional weight in class i Ω Spatial domain of the image N Number of segmentation classes θ Trainable parameters of the CNN Pvi (θ ) Softmax output for the vth pixel value of the class ith T One-hot vector of the ground truth iv CHAPTER 1.1 Motivation for Participating in Medical Image Segmentation Challenges Medical image segmentation challenges provide a unique opportunity for re- searchers and practitioners to address critical problems in healthcare through the development of advanced computational techniques. By participating in these challenges, participants aim to contribute to the improvement of diagnosis, treat- ment planning, and patient care.
In this section, I discuss the motivations behind participating in four specific medical image segmentation challenges: the Lesion Boundary Segmentation challenge on the ISIC-2018 dataset, the dermoscopic PH2 database, the 2017 MICCAI sub-challenge on automatic cardiac diagnosis bench- mark, and the 6-month infant brain MRI Segmentation (iSeg) benchmark. Skin cancer is a prevalent and potentially deadly condition that demands early and accurate detection. The ISIC-2018 challenge [1, 2] and the PH2 challenge [3] focus on segmenting skin lesions, aiming to improve the accuracy and efficiency of diagnosis. The motivation to participate in this challenge arises from the urgent need to develop automated segmentation methods that can assist dermatologists in identifying and diagnosing skin cancer.
Successful segmentation of lesion bound- aries can enable more accurate diagnosis and early intervention, ultimately enhanc- ing patient outcomes. Participating in this challenge provides an opportunity to contribute to dermatological research, develop advanced segmentation techniques, and potentially revolutionize skin cancer diagnosis. Cardiovascular diseases are a leading cause of mortality globally. The MIC- CAI sub-challenge on automatic cardiac diagnosis [4] addresses the need for ac- curate cardiac segmentation to aid in diagnosing heart conditions.
The motivation to participate in this challenge stems from the potential to advance cardiac imag- ing and diagnosis through automated segmentation methods. Precise segmentation of cardiac structures can assist cardiologists in assessing heart function, identify- ing anomalies, and guiding treatment decisions. By participating in this challenge, researchers can collaborate with experts in cardiology, contribute to cutting-edge medical research, and develop solutions that have a direct impact on patient care. Segmentation of infant brain MRI scans is crucial for studying early brain development and identifying abnormalities.
The iSeg benchmark challenge [5] fo- cuses on accurate segmentation of infant brain structures, aiding in early detection of neurological disorders. The motivation to participate in this challenge lies in the potential to contribute to pediatric neuroimaging and improve the understanding of infant brain development. Accurate segmentation of brain structures can assist clinicians and researchers in diagnosing conditions and monitoring developmental 1 milestones. Participation in the iSeg benchmark offers the chance to advance pedi- atric imaging, collaborate with experts in the field, and create tools that facilitate early intervention and improved patient outcomes.
In conclusion, participating in medical image segmentation challenges pro- vides a unique avenue to address critical healthcare challenges. The motivations behind participating in these challenges range from improving diagnosis accuracy and treatment planning to collaborating with experts and contributing to cutting- edge medical research. These challenges offer a platform for researchers to develop and showcase innovative solutions that have the potential to revolutionize health- care practices and enhance patient care.2 Advancements in Medical Image Segmentation and Innovative Approaches Image segmentation is a pivotal and challenging topic in the field of computer vision [6]. Its objective is to partition an image in a way that accurately locates, identifies, and quantifies objects.
This process holds crucial importance in medical imaging, supporting additional clinical analysis, diagnosis, therapy planning, and disease progression measurement.