NGUYỄN ANH DŨNG AIOT BASED NEURAL DECODING AND NEUROFEEDBACK FOR A COGNITIVE TRAINING ACCELERATION LUẬN VĂN THẠC SĨ ĐIỆN TỬ, NĂNG LƯỢNG ĐIỆN, TỰ ĐỘNG HÓA CHUYÊN NGÀNH KỸ THUẬT TRUYỀN THÔNG VÀ DỮ LIỆU NGƯỜI HƯỚNG DẪN KHOA HỌC GS. Nguyễn Văn Tâm TS. Lê Vũ Hà GS. Jocelyn FIORINA Paris - 2024 Wavelet-Based Pre-processing for Improved Analysis and Classification of EEG Signals Master thesis of Paris-Saclay University and VNU University of Engineering and Technology Specialization: M2 Data and Communication Engineering Research unit: Département Communications et Électronique Télécom Paris - IP Paris Thesis presented at Paris-Saclay, on 29 March 2024 Nguyen Anh Dung Composition of Jury Arnaud BOURNEL Paris-Saclay University Président Pierre DUHAMEL CNRS, CentraleSupelec, Paris-Saclay University Rapporteur Van Tam NGUYEN COMELEC, Telecom Paris Directeur Vu Ha LE VNU University of Engineering and Technology Co-directeur Linh Trung NGUYEN VNU University of Engineering and Technology Examiner Supervisors of the thesis Master Thesis Van Tam NGUYEN COMELEC, Telecom Paris Supervisor.
Vu Ha LE VNU University of Engineering and Technology Co-supervisor. Jocelyn FIORINA CentraleSupelec, Paris-Saclay University Co-supervisor. Acknowledgements I express my deepest appreciation to Prof. Van-Tam NGUYEN and Dr.
LE Vu Ha who provided me the precious knowledge and essential skills, helped me to find, choose the topic, and supported me in the subject. I would also like to express my profound gratitude to Prof. Van-Tam NGUYEN and the Department of Communications and Electronics (COMELEC) for providing me with the necessary equipment to facilitate the research process essential for the completion of my thesis. My sincere thanks also go to lecturers in the University of Paris-Saclay and Vietnam National University - University of Engineering and Technology who taught many valuable subjects in my university’s course.
There are the base knowledge for my thesis. I also would like to sincerely thank Prof. Van-Tam NGUYEN at Telecom Paris, Prof. Pierre DUHAMEL at CentraleSupélec - Université Paris-Saclay, Dr.
LE Vu Ha, Assoc. NGUYEN Linh Trung and Dr. NGUYEN Hong Thinh at VNU - University of Engineering and Technology helped me a lot during the time of applying for the joint master’s program and preparing my necessary documents to intern in France. I am fortunate to have been a member of the Department of Communications and Electronics (COMELEC).
This is a memorable time for me to work in a very professional environment in France, with beloved teachers and friends who always support me in my research work as well as in social life. Last, but not least, my warm and heartfelt thanks go to my family for their tremendous support and hope they had given to me. In the process of developing and completing this Master Thesis, due to many limitations in experience and time, mistakes are inevitable. I look forward to receiving the sympathy and contributions of the lecturers in the Council to help me have a better overview of the thesis.
Massy, March 21st 2024 Student NGUYEN Anh Dung Authorship I solemnly declare that my thesis, titled ’Wavelet-Based Pre-processing for Improved Analysis and Classification of EEG Signals’, is my own research work conducted under the guidance of Prof. Van Tam NGUYEN, Prof. Vu Ha LE and Prof. The sources used in the thesis are explicitly mentioned in the reference section, with proper citations.
The data and results presented in the thesis are entirely truthful, and there is no copying from the works of others. If any discrepancies are found, I take full responsibility and am subject to any disciplinary actions imposed by the university., 2024 Student Nguyen Anh Dung List of Abbreviations NF Neural feedback AIoT Artificial Intelligence of Things EEG Electroencephalography BCI Brain-Computer Interface STFT Short-Time Fourier Transform WT Wavelet Transform DWT Discrete Wavelet Transform CWT Continuous Wavelet Transform kNN k-Nearest Neighbor DBSCAN Density-Based Spatial Clustering of Applications with Noise LOO Leave-One-Out GBDT Gradient Boosting Decision Trees RF Random Forest XGB eXtreme Gradient Boosting SVM Support Vectore Machine NN Neural Network DNN Deep Neural Network CNN Convolutional Neural Network LSTM Long-Short Term Memory 1 Contents 1 Introduction 8 1.2 Pre-processing and feature extraction .3 Classification methods .3 Motivation of Study. 17 2 Pre-processing and Feature extraction 18 2.1 Pre-processing methods .3 Discrete Wavelet Transform .4 Continuous Wavelet Transform .1 Modification of the EEG Headset. 29 2 3 Classification method 31 3.2 eXtreme Gradient Boosting .3 Support Vector Machine .4 Deep Neural Network .5 Convolutional Neural Networks .6 Long-Short Term Memory.
42 4 Experiments and Results 44 4.1 Discrete Wavelet Transform .2 Continuous Wavelet Transform. 54 5 Conclusions and Future works 58 5.2 Discrete Wavelet Transform .3 Continuous Wavelet Transform. 60 A Random Forest result for Subject Specific 64 B XGBoost result for Subject Specific 71 C Support Vector Machine result for Subject Specific 78 D Random Forest result for Leave-One-Out 85 E XGBoost result for Leave-One-Out 92 F Support Vector Machine result for Leave-One-Out 99 G Random Forest result for Common subject 106 H DNN4 result for Subject Specific 113 I DNN result for Common subject 115 3 J DNN4 result for Leave-One-Out 117 K DNN result for Leave-One-Out 119 4 List of Figures 1.1 Brain lobes and sub-areas [12] .2 fMRI image and device .5 Feature vector shape .1 The resolution at time-frequency domain .2 The resolution at time-frequency domain on signal 2.3 Some basic wavelet function Wasilewski [26] .4 Wavelet Transform method division diagram .6 Decompose low-pass and high-pass filter for "Daubechies 4" [26] .8 CWT spectrogram created from by convolution with set of wavelet fucntion daughter .9 Electrode locations of International 10-20 system .10 Underfitting and Overfitting .2 Random Forest structure [10] .3 Basic Boosting structure .4 eXtreme Gradient Boosting structure .6 Soft margin with 3 type of data point that not in safe area [2] .10 Basic Neural Network structure .11 CNN basic structure [16] .12 Recurrent neural network structure 1 .13 Long-Short Term Memory cell at one time 2 .1 Mean accuracy with Subject-Specific paradigm with multiple classifier with window shift 256 samples .2 Accuracy with Common subject paradigm with multiple classifier with window shift 256 samples .3 Mean accuracy with Leave-One-Out paradigm with multiple classifier with window shift 256 samples .4 Mean accuracy with Subject-Specific paradigm with multiple classifier with window shift 384 samples .5 Accuracy with Common subject paradigm with multiple classifier with window shift 384 samples .6 Mean accuracy with Leave-One-Out paradigm with multiple classifier with window shift 384 samples .7 Mean accuracy on LOO paradigm with Decompose level 6 and Window shift 256 samples .8 Mean accuracy on LOO paradigm with Decompose level 6 and Window time 24 .9 Mean accuracy on LOO paradigm with Window time 24 seconds and Window shift 256 samples .10 Train accuracy and loss on whole process .11 Test accuracy and loss on whole process .12 Train accuracy and loss on whole process with window 24 seconds and 100 epochs .13 Test accuracy and loss on whole process with window 24 seconds and 100 epochs. 56 6 List of Tables 1.1 Brain wave frequencies .2 A Snippet of the Raw Data .1 Decompose filter coefficients .1 Best result on the modified Subject-Specific and Common subject .2 Best result on the modified LOO .3 DWT accuracy using RF heatmap .4 DWT accuracy using XGB heatmap .5 DWT accuracy using SVM heatmap .6 DNN4 for DWT 2D feature .7 DNN6 for DWT 2D feature .8 CNN for DWT 2D feature 256 samples .9 CNN for DWT 2D feature 384 samples .10 LSTM for DWT 2D feature 256 samples .11 LSTM for DWT 2D feature 384 samples .12 CNN result for CWT with learning rate 0.13 CNN result for CWT with learning rate 0.14 CNN result for CWT with learning rate 0.15 CNN result for CWT with learning rate 0.16 Table of feature generalizability results with LOO paradigm .1 Problem Statement Emotional Intelligence is one the most important part of our lives.
It can affect our physical and mental health, community relations, learning and working performance [20]. In other words, It is a term used to describe a person’s ability to manage, respond based on their experience and promote the development of their concentration and vigilance. Attention is one fundamental component of cognitive basis [18]. This process requires careful selection of information and avoidance of distractions.
To maintaining high performance, this cognitive process is essentially, which is directed by goals primarily located in the prefrontal cortex [22]. The emotional content that we get from stimuli will be presented to the sensory system which is the principal indicator of the importance for these stimuli [14]. Therefore, emotion and attention has a strongly connective. In the other hand, attention also help people in improving skill and memory.
Information we get from experience will be chosen by attention to remember carefully and store it in memory. The allocation of cognitive resources is limited and attention is the most important factor for people to manage and divide it. Attention can be improved with Neural Feedback (NF) which is a form of biofeedback. To facilitate the self- regulation of neural substrates thought to underlie a particular behavior or pathology, neural activity will be mea- sured and presented directly to the participant through sensory channels [21].
NF can be also used to improve neuroplasticity, which is the ability of neural networks in the brain to change through growth and reorganization when new things such as environment factor, knowledge [4]. NF training causes unique neural alterations in grey and white matter anatomy corresponding to the trained brain circuit and concomitant behavioral changes [21]. Based on the concept of neuroplasticity, cognitive training appear to enhance cognitive abilities such as memory, attention [8]. Cognitive training can be defined as a series of actions that challenge and stimulate the brain, with the aim of improving the efficiency of various cognitive functions.
It has long-term effects, even on healthy persons and 8 those with cognitive impairment. Therefore, esigning cognitive training techniques is required to reduce cognitive deterioration in the elderly [14]. Cognitive training can be accomplished by detecting human physiological signals, doing pre-processing and processing with TinyML, and using AI models to develop appropriate NF systems with precision, high accuracy, and short response time. This might be considered the creation of AIoT-based neurofeedback devices that are very efficient and have limited power sources.
Due to constraints on energy resources, applied machine learning models will prioritize compactness, emphasizing a small parameter count. In the realm of emotional signal processing, the utilization of machine learning models necessitates the implementation of various preprocessing methods tailored to enhance the efficiency of deep neural network (DNN) models.1 Brain structure The brain is a complex structure composed of 60% white matter and the remaining gray matter. It primarily controls memory, thinking, and bodily states such as emotions, touch, and hunger, as well as the body’s regulatory processes. The brain is divided into 3 main parts: cerebrum, brainstem and cerebellum.
The cerebrum, located at the front of the brain, is the largest region and consists of a large area of gray matter covering the white matter in the middle. This region plays a vital role in regulating temperature, actions, receiving information from the senses, and processing information to produce emotions, thoughts, speech, evaluation, and problem-solving.The large area of gray matter in this region is also known as the cerebral cortex. Due to its numerous folds covering the brain, the surface area of this gray matter region is quite large and accounts for half the weight of the brain. The cerebral cortex is divided into two large cerebral hemispheres and each hemisphere controls the opposite side of the body, with the right hemisphere controlling the left body part and the left hemisphere controlling the right body part [13].
The brainstem is the central region of the brain that connects the cerebrum to the spinal cord. It comprises three main parts: the midbrain, the pons, and the medulla [13].