LÊ HUY RATE, DISTORTION AND CLASSIFICATION TRADEOFF LUẬN VĂN THẠC SĨ NGÀNH ĐIỆN TỬ, NĂNG LƯỢNG ĐIỆN, TỰ ĐỘNG HÓA CHUYÊN NGÀNH KỸ THUẬT TRUYỀN THÔNG VÀ DỮ LIỆU NGƯỜI HƯỚNG DẪN KHOA HỌC GS. ARMELLE WAUTIER GS. PIERRE DUHAMEL TS. ĐỖ THANH HÀ PARIS, NĂM 2024 RATE-DISTORTION AND CLASSIFICATION TRADEOFF Master thesis of Paris-Saclay University and VNU University of Engineering and Technology Specialization: M2 Data and Communication Engineering Research unit: Laboratory of Signals and Systems CentraleSupélec, Paris-Saclay University Thesis presented in France, on 21 December 2023 LE Huy Committee Arnaud BOURNEL Paris-Saclay University Chairman Anissa MOKRAOUI L2TI - Information Processing and Transport Labo- Reporter ratory - Paris-13 University Armelle WAUTIER L2S, CNRS - CentraleSupélec - Paris-Saclay Univer- Director sity Pierre DUHAMEL L2S, CNRS - CentraleSupélec - Paris-Saclay Univer- Examiner sity NGUYEN Linh Trung VNU University of Engineering and Technology Examiner 0DVWHU7KHVLV Thesis Supervision Armelle WAUTIER L2S, CNRS - CentraleSupélec - Paris-Saclay Univer- Supervisor.
sity Pierre DUHAMEL L2S, CNRS - CentraleSupélec - Paris-Saclay Univer- Co-supervisor. sity DO Thanh Ha VNU University of Science Co-supervisor. Acknowledgement As my Master Thesis is a success, I express my deepest appreciation to Prof. Armelle WAUTIER, Prof.
Pierre DUHAMEL and Dr. DO Thanh Ha who provided me the precious knowledge and essential skills, helped me to find, choose the topic, and supported me conscientiously in the subject. My sincere thanks also to lecturers in the University of Paris-Saclay and Vietnam National University - University of Engineering and Technology taught many valu- able subjects in my university’s course. There are the base knowledge for my thesis.
I also would like to sincerely thank Prof. Arnaud BOURNEL in Université Paris-Saclay, Prof. Armelle WAUTIER, Prof. Pierre DUHAMEL in CentraleSupélec - Université Paris-Saclay, Prof.
NGUYEN Linh Trung and Dr. NGUYEN Hong Thinh in VNU - University of Engineering and Technology, Dr. DO Thanh Ha in VNU - University of Science helped me a lot during the time of applying for the joint master’s program and preparing my necessary documents to intern in France. I am fortunate to have been a member of the Laboratoire des Signaux et Systèmes, L2S - CentraleSupélec - Université Paris-Saclay.
This is a memorable time for me to work in a very professional environment in France, with beloved teachers and friends who always support me in my research work as well as in social life. Last, but not least, my warm and heartfelt thanks go to my family for their tremendous support and hope they had given to me. In the process of developing and completing this Master Thesis, due to many limitations in experience and time, mistakes are inevitable. I look forward to receiving the sympathy and contributions of the lecturers in the Council to help me have a better overview of the thesis.
Gif-Sur-Yvette, November 26th 2023. Student LE Huy I solemnly declare that my thesis, titled ”Rate-Distortion and Classifica- tion tradeo↵”, is my own research work conducted under the guidance of Prof. Armelle Wautier and Prof. Pierre Duhamel and Dr.
DO Thanh Ha. The sources used in the thesis are explicitly mentioned in the reference section, with proper citations. The data and results presented in the thesis are entirely truthful, and there is no copying from the works of others. If any discrepancies are found, I take full responsibility and am subject to any disciplinary actions imposed by the university.
Gif-Sur-Yvette, November 26th, 2023 Student Le Huy Contents 1 Theoretical Framework 6 1.1 Rate in gray-scale image .2 Rate in latent space for deep learning encoders .4 Distortion-Perception tradeo↵ for given rate .5 Rate-Distortion-Perception tradeo↵ .1 Overview the structure of Wasserstein Generative Adversarial Networks .4 Tuning coefficients for optimization on training in WGAN model .2 Classification performance metrics .4 Perception with four Probability Density Functions for coefficients in Haar decomposition .5 Link between perception index criterion and Haar transform .6 Proposed encoder and decoder architecture .2 Pre-process of dataset .3 Proposed encoder/decoder evaluation .4 Classifier algorithm validation .5 Encoder/Decoder performance with classification criteria. 61 5 Conclusion and future works 65 2 Abstract The purpose of the Master thesis is to improve the classification performance upon reception of an image, for a given rate and distortion. It builds on the work of Blau and Michaeli [BM18] [BM19] to integrate perceptual quality in coding by introducing the divergence between input and output signal distributions as a criterion, thereby redefining the rate/distortion tradeo↵ to include perception. My research extends this by incorporating image gradient statistics for enhanced segmentation in compressed images, therefore resulting in a slight modifica- tion of the Rate/Distortion/Perception model for improved classification performance.
This modification is based on the use of a 2D Haar transform, which allows to include the diver- gence between the high frequency components of the original and reconstructed images in the criterion to be optimized. Central to my approach is the use of Machine Learning, especially Wasserstein Generative Adversarial Networks (WGANs), marking a significant integration of traditional coding techniques with contemporary AI innovations. 3 Introduction This thesis sets out to examine the classification performance of source coding with two tradeo↵: Rate, Distortion, but follows the same lines as was proposed in [BM19] for rate, distortion, and perception. Traditional image coding methods, rooted in the principles of lossy source coding from information theory [Sha48], often apply perceptual elements in an inconsistent manner, lacking a cohesive theoretical basis.
The work in [BM19] intends to address this disparity by integrating recent developments in Machine Learning, aiming to harmonize perceptual quality with distortion measurements to create more advanced and theoretically sound coding algorithms. My target of the master thesis is classification performance. The papers of Blau and Michaeli [BM18] [BM19] in 2019 had proposed the rate, distortion and perception tradeo↵. Amazingly, they also proposed to include some Deep feature based distortion [BM18][BM19] for enhancing the semantic similarity of the original and reconstructed image.
In their papers, however, they do not really follow this line, and do not follow the direction in classification. My contribution will replace the perceptual by classification to do the classification performance and evaluate it. In 2018 Blau and Michaeli [BM18] introduced a method to incorporate perceptual qual- ity in image coding by assessing the di↵erence between the distributions of the input and output signals, typically through a divergence measure. This idea is crucial in redefining the classic rate/distortion tradeo↵ to include a perceptual dimension, further explored in their subsequent 2019 paper [BM19].
In this paper, they demonstrate the existence of a Rate/Distortion/Perception tradeo↵ that can be accurately assessed using simple examples. Our research aims to build upon this foundation by considering the statistics of image gradients as a criterion for maintaining segmentation capabilities in compressed images. This advancement could lead to initial steps in the facilitation of e↵ective computational tasks on these images, like proficient classification, while maintaining a balance in the rate/distortion tradeo↵s. Moreover, we plan to explore how the Rate/Distortion/Perception tradeo↵ model can be tailored to optimally determine the best classification/rate/distortion balance, steering towards a more holistic.
Although this topic is closely linked to source coding, the primary tool in this research will be Machine Learning, with a special focus on Wasserstein Generative Adversarial Networks (WGANs), as used in Blau and Michaeli’s original research. This research represents a significant step in merging traditional coding techniques with the latest advancements in Machine Learning to innovate in the field of image coding. In addition, we proposed a new image dataset applied 2-Dimension Discrete Haar transform [Mal89]. The goal of this method is to achieve a sparse representation that highlights the key features of the image, rather than maintaining a comprehensive representation from the original image.
4 Contents Subsequently, the classification performance will be compared between the original dataset and the dataset processed with the 2D Haar transform. On this master thesis, there are four main chapters: • Chapter 1: Theoretical Framework: Some knowledge in rate, distortion and perception tradeo↵ • Chapter 2: Methodology: Rate-Distortion-Perception in the context of WGAN-based image restoration • Chapter 3: My contribution: Classification problem, performance and Haar transform • Chapter 4: Experiment results: Evaluate result of reconstruct images and benchmark classification performance • Chapter 5: Conclusion and Future Work: Summarize the results achieved and direc- tions for further development Figure 1: Flowchart of the initial approach The Figure 1 represents a flowchart of a proposed image restoration with classification process. The process begins with an input MNIST dataset and which are fed into a WGAN model. The WGAN model’s output includes a reconstructed MNIST dataset.
In addition, these reconstructed datasets undergo a comparison of distortion and perception at a given rate, marked by a dashed-line box. The reconstructed MNIST dataset is then input into a classification model, the performance of which is subsequently benchmarked and evaluated. University of Paris-Saclay and VNU-UET 5 Chapter 1 Theoretical Framework 1.1 Rate in gray-scale image The MNIST (Modified National Institute of Standards and Technology database) handwrit- ing dataset utilizes grayscale images with an 8-bit depth, meaning each pixel is represented by 8 bits. This bit depth, crucial in digital imaging, dictates the range of tones a pixel can exhibit, impacting the image’s overall color range.
In digital imagery, there are bitonal, grayscale, and color images. The term ’bits per pixel’ (bpp) indicates the bit depth, influ- encing the number of colors an image can display, calculated as 2bpp. MNIST’s images, at 28 ⇥ 28 ⇥ 1 pixels, use a single channel for grayscale, allowing for 256 shades (from black to white), with the grayscale conveying information through light intensity only.1 give an example of image in MNIST data. Base on the source coding theorem from Shannon [Sha48], The entropy H(X) gives the minimum number of bits needed to encode value x, on average, in such a way that no information is lost.
It is also the maximum number of bits needed to compress optimally source: X H(X) = p(X = x) ⇥ log2 p(X = x), (1.1) x2X where X is random variable Figure 1.1: Example of image from the MNIST dataset with size 28 ⇥ 28 ⇥ 1 and 8 bits per pixel 6 Chapter 1. Theoretical Framework Figure 1.2: Overall the compression diagram This objective of lossy compression is to further reduce the rate. On the Figure 1.2, the original image is denoted by X will be encoded in the encoder block. The reconstructed image X̂ will decode in decoder block.
Classical image compression codec such as JPEG (Joint Photographic Experts Group) in 1992 [Wal92] uses the discrete cosine transform (DCT), JPEG 2000 in 2000 [SCE01] uses transform, Better Portable Graphics (BPG) in 2014 [AMK15] uses DCT. From the classical approach, they concern only on minimizing distortion at any given rate. Moreover, the machine learning based codecs for lossy compression to reduce coding rate, keep better result and focus more about the perception with distortion and rate. Many state-of-the-art deep compression approach rely on context models to capture the distribution of the bit stream [BLS16] [Tod+17] [Li+18] [RB17] [Men+18] 1.2 Rate in latent space for deep learning encoders Dimension dim Quantization Level L W Ŵ X Encoder f Quantizer q Decoder g X̂ Figure 1.3: Architecture of encoder/decoder for deep image compression In the framework of using a deep learning compression model [Agu+19], coding rate is represented in the latent space.
The encoder’s objective is to compress the original image (input image) into a latent space vector or matrix of a given dimension. The latent space is an abstract multi-dimensional domain, which contains feature values In the domain of deep image compression, Figure 1.3, show the principle of deep learning compression comprises an encoder f , a decoder g, and a finite quantizer q. The encoder transforms an input image X into a latent feature map W with a dimension denoted by dim, which is subsequently quantized into L discrete levels, represented by a set C = {c1 ,. , cL } ✓ R, yielding the quantized representation Ŵ = q(f (X)).