VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY FACULTY OF COMPUTER SCIENCE NGUYEN PHUC DAT - HUYNH MINH TRI THESIS BOUNDARY-AWARE NETWORK FOR THORACIC DISEASES SEGMENTATION HONORS BACHELOR IN COMPUTER SCIENCE HO CHI MINH CITY,2021 VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY FACULTY OF COMPUTER SCIENCE NGUYEN PHUC DAT - 18520573 HUYNH MINH TRI - 18520176 THESIS BOUNDARY-AWARE NETWORK FOR THORACIC DISEASES SEGMENTATION HONORS BACHELOR IN COMPUTER SCIENCE THESIS ADVISOR NGUYEN VINH TIEP, Ph. HO CHI MINH CITY,2021 ASSESSMENT COMMITTEE The Assessment Committee is established under the Decision. „ date " by Rector of the University of Information Technology. Le Dinh Duy — Chairman.
Nguyen Thanh Son — Secretary. Le Minh Hung — Member. Acknowledgement Foremost, we would like to express our sincere gratitude to our advisor PhD. Nguyen Vinh Tiep for the support of our thesis, for his patience, motivation, enthusiasm, and immerse knowledge.
His guidance help us in all the time of research and writing of this thesis. Besides our advisor, we would like to thank Mr.Nguyen Thanh Danh, a brother who helps us a lot in our work, for all the comments, the explanations and the dedication in helping our thesis becoming better and better everyday Besides, we would like to thank the Faculty of Computer Science - University of Information Technology where we have been prepared enough knowledge to pursuing the field of Computer Science research. We also thank Multi Media Laboratory (MMLab-UIT) for supporting us the research environment and cutting-edge devices throughout the university journey. Last but not least, we would like to thank our family for supporting us, encourage us getting through the hard time since we have to finish our work during the pandemic.
ABSTRACT In computer vision, image segmentation is a fundamental but challenging task aiming to assign a label to each pixel in the image. With the rapid development of deep learning, image segmentation archives impressive results in a variety of fields such as autonomous vehicles, medical systems, photo editors, and so on. In medical, image segmentation with high results can provide a reliable basis for further clinical analysis and disease diagnosis. The X-ray image is a special type of image in medical which has a different number of channels and a range of pixel values to normal images.
Besides, the contrast level between foreground and the background in the X-ray image is significant low. Although there are many studies on the X-ray images, the majority of these research mostly about the classification and detection task. Therefore, segmenting diseases in Chest X-ray images is difficult for both experimental doctors and researchers in the computer vision field. To address this problem, radiologists often rely on the boundary and shape of the diseases to detect them.
Many works has done on improving the result of this problem based on this idea by guiding their deep learning model focuses on the boundary of the diseases and the signed distance map of the images. In this work our first contribution is to summarize the latest developments of these boundary-aware methods. The second contribution is that we evaluated the efficiency of these techniques.21 Lack of annotated data.2 Abnormal findings in chest X-rays. Indistinguishable boundary between normal and abnormal areas in Chest X-rays 2.
eeen 6 13 Objectives Ag7. Ặ HQ ho 9 2 Related Work 11 2.1 Overview of Image Segmentation .4 Boundary-aware Image Segmentation.3 Medical Image Segmentation.1 Overview of Medical Image Segmentation.2 Thoracic diseases segmentation 20 2. Boundary-aware in Thoracic diseases segmentation .4 Loss functions in Image Segmentation.1 Distribution-based Loss .2 Region-based Loss 2.3 Boundary-based Loss.1 Overview proposed methods.2 Signed distance map and boundary-based loss funetions.2 Boundary-based loss functions 26 3.2 Into BMask R-CNN .4 Combine loss functions method. ee 36 4 Experiments 38 41 Overview.2 Dataset and Evaluation.
eee eee 38 43 Results. <o-" xứ ốc TW.1 U-Net with boundary loss functions.2 U-Net with Signed Distance Map. Combine loss function methods.21 Modified BMask R-CNN with Signed Distance Map. ee ee 48 5 Conclusion 50 5.ee 51 References 53 List of Figures 1.1 Brain tumor segmentation.2 Comparison of a high- and low-resolution chest X-ray based on a pneu- mothorax.
(a) shows the original chest X-ray in the full image size of 2828 x 2320 pixels. In (b), two areas of (a) are shown, magnified by a factor of ten. The yellow arrows point to the edge of the pleura, which indicates the pneumothorax. For comparison, (c) shows (a) downscaled by bilinear interpolation to an image size of 256 x 256 pixels.
(d) shows the same magnified areas as (b) and the pleura edge is no longer visible 8 1.1 Semantic Segmentation and Instance Segmentation .2 Fully Convolutional Networks .4 General framework for Classification of Mask Proposals Techniques 2.5 General framework for Detection Followed by Segmentation Techniques 17 2.6 General framework for Labelling Pixels Followed by Clustering Techniques 17 2.7 General framework for Dense sliding window methods Techniques 18 2.8 Dice coefficient in set view.9 A schematic showing the Hausdorff Distance between points sets X and Y .2 The different between Binary mask and SDM. In this figure, we clip the range value of SDM from-ltol. The evaluating of the changes between two boundaries. (a) An example of a 2D ground truth and predicted segmentations.
(b) The SDM dy with dp overlaid in red. The white circle shows the location of the largest dy, which corresponds to hd(dp, 6q). ee The Shared output model architeetire.7 The Multi-heads model architeeture.8 The Reconstruction branch model architecture.9 The Reconstruction branch model with fusion head architecture.10 The Overall architecture of Boundary-preserving Mask R-CNN (BMask R-CNN) .11 Ground truth of the boundary branch a) Original BMask R-CNN ground truth b) Signed Distance Function ground truth c) Distance Transform Map ground truth.12 The modified architecture of Boundary-preserving Mask R-CNN (BMask R-CNN) .1 Examples in ChestX-det .2 Visualization of segment mask results of models in section 4. The columns’ names is labels of segmented area.
The rows’ name is the models’ name.3 Visualization of segment mask results of models in section 4. The columns’ names is labels of segmented area. The rows’ name is the models’ nama ra.4 Visualization of signed distance map results of models in section 4. The columns’ names is labels of segmented area.
The rows’ name is the models’ name. In case of signed distance maps, each class have each individual maps, so we just visualize one of classes appearing in images. 46 45 Visualization of segment mask results of models in section 4. The columns’ names is labels of segmented area.
The rows’ name is the models’ name. HH va List of 'Tables 41 The statistics of abnormalities.2 Results of Shared output model.3 Results of Multi-heads model and Reconstruction branch model .4 Results of applying combine loss method in Reconstruction branch with fusion head. ee 44 Result of test dataset on ChestX-Det. AT vi Chapter 1 Introduction The primary theme of this thesis is image segmentation applied on medical images.
This chapter will begin by outlining the basic problem of segmentation and motivate its importance in many applications. Modern medical imaging modalities generate larger and larger images which simply cannot be examined manually. This drives the development of more efficient and robust image analysis methods, tailored to the problems encountered in medical images. The aim and motivation of this thesis in Section 1.2 are directed towards the particular problem of segmenting thoracic diseases, boundary-aware segmentation to be more specific.
However, the generality of the problem can lead to potential impacts also in other areas of image analysis.1 Overview Image segmentation is the problem of partitioning an image in a semantically meaningful way. This vague definition implies the generality of the problem - segmentation can be found in any image-driven process, e. fingerprint/text/face recognition, detection of anomalies in industrial pipelines, tracking of moving people/cars/airplanes, etc. For many applications, segmentation reduces to finding an object in an image.
This involves partitioning the image into two class of regions - either object or background. Segmentation is taking place naturally in the human visual system. We are experts on detecting patterns, lines, edges and shapes, and making decisions based upon the visual information. At the same time, we are overwhelmed by the amount of image information that can be captured by today’s technology.
It is simply not feasible in practice to manually process all the images (or it would be very expensive, and boring, to do so). Instead, we design algorithms which look for certain patterns and objects of interest and put them to our attention. For example, a recent popular application is to search and match known faces in your photo library which makes it possible to automatically generate photo collections with a certain person. An important part of this application is to segment the image into “face” and “background”.
This can be done in a number of ways, and it is well accepted that no general purpose segmentation algorithm exists, or that it ever will be invented. Thus, when designing a segmentation algorithm, the application is always of primary focus: Should we segment the image based on edges, lines, circles, faces, cats or dogs? 1.1 Practical Context An interesting source of images is the medical field. Here, imaging modalities such as CT (Computed Tomography), MRI (Magnetic Resonance Imaging), PET (Positron Emission Tomography) etc. generate a huge amount of image information.
Not only does the size and resolution of the images grow with improved technology, also the number of dimensions increase. Previously, medical staff studied two-dimensional images produced by X-ray. Now, three-dimensional image volumes are common in everyday practice. Even four-dimensional data (three-dimensional images changing over time, i.
movies) is often used. This increase in size and dimensional provides major technical challenges as well as cognitive. How do we store and transmit all this data, and how can we look at it and find relevant information? This is where automatic, or semi-automatic, algorithms are of interest. In the best of worlds, we would like to have algorithms which can automatically detect diseases, lesions and tumors, and highlight their locations in the large pile of images.
But another complication arises, we also have to trust the results of the algorithms. This is especially important in medical applications - we don’t want the algorithms to signal false alarms, and we certainly don’t want them to miss fatal diseases. Therefore, developing algorithms for medical image analysis requires thorough validation studies to make the results usable in practice. This adds another dimension to the research process which involves communication between two different worlds - the patient-centered medical world, and the computer centered technical world.
The symbiosis between these worlds are rare to find and it requires significant efforts from both sides to join on a common goal.2 Problem Definition Driven by the revolution of big data and computing power, deep learning techniques have shown their ability to deliver outstanding performances in different fields where machines become more suitable than humans dealing with a huge amount of data in a comprehensive way. The medical field is one of the fields where deep learning had the largest impact. Last years have witnessed a revolution in medical imaging that has broken paradigms and seems to open new possibilities in the field. Recent studies estimate that medical images currently account for at least 90% of medical data, which is an overwhelming amount on human scale.
The detection, classification and segmentation of specific medical patterns, e.g organs, tissues, tumors are the main key steps for diagnosis, treatment. or monitoring of different diseases progression. For instance, an accurate delineation of one or multiple structures is essential to derive important clinical parameters such as ventricular volume, ejection fraction, and myocardial mass for cardiac images or lesion volume for brain images. However, the manual segmentation of these structures is complex, time consuming and requires multiple expert opinions.