VIETNAM NATIONAL UNIVERSITY, HANOI INTERNATIONAL SCHOOL GRADUATION PROJECT DIGITAL SIGNAL PROCESSING COMBINED WITH MACHINE LEARNING IN DIABETES DIAGNOSIS Student’s name Do Cong Tuan Hanoi – Year 2024 VIETNAM NATIONAL UNIVERSITY, HANOI INTERNATIONAL SCHOOL GRADUATION PROJECT DIGITAL SIGNAL PROCESSING COMBINED WITH MACHINE LEARNING IN DIABETES DIAGNOSIS SUPERVISOR: Assoc. Nguyen Thanh Tung STUDENT: Do Cong Tuan STUDENT ID: 19071639 COHORT: QH2019 SUBJECT CODE: INS401401 MAJOR: Informatics and Computer Engineering Hanoi – Year 2024 2 THANKS I would like to express my deepest gratitude to Assoc. Nguyen Thanh Tung, who taught, supported, and guided me throughout the process of completing this graduation project. I would like to sincerely thank all the lecturers at International School - Vietnam National University, Hanoi who have dedicatedly taught me and provided me with a lot of useful knowledge, both academic and soft skills.
This helps me a lot in my personal development and my future career. After a period of accumulating knowledge, this project is the final test for me, giving me the opportunity to evaluate myself and my abilities. I was truly able to apply all the knowledge I learned from lecturers, books, documents, and many other sources. It also helps me really review my knowledge.
3 COMMITMENT The independence of research in academia, as demonstrated in a report, plays a crucial role in establishing the author's credibility and separation from industry standards. A notable example is the evaluation of plagiarism levels, where a deviation from acceptable ratios may indicate intellectual property theft, suggesting the unauthorized use of someone else's work or a lack of originality in the new work. Such circumstances could imply a lack of significant advancements compared to existing publications, making the new work of limited or no scholarly value. Additionally, both Vietnamese and global legal frameworks offer clear definitions and punitive measures as effective deterrents against such infringements.
As a student deeply involved in an academic environment, I am aware of the negative effects and consequences of violations. Therefore, I would like to affirm the following commitments to the school administration and the legal system: I confirm that my graduation project, titled “Digital Signal Processing Combined with Machine Learning in Diabetes Diagnosis” is the result of my independent research efforts and has not been used to earn any degrees or certificates from any other educational institution. Following ethical research principles, I acknowledge all sources used in this project. Each citation is diligently identified and annotated in accordance with prevailing standards, specifically adhering to the APA format.
I declare that all information presented here is truthful and accurate to the best of my knowledge and belief. In the event of any identified violations, I willingly accept full responsibility and acknowledge the potential sanctions imposed by the school and legal authorities. Hanoi, date 17 month 06 year 2024 Student (Signature and Full name) 4 ABTRACT The graduation report focuses on implementing a signal processing software library based on the Improved Modified Polynomial principles method, derived from the Polynomial Fitting Method. The data processed is then used to train a machine learning model to compare different adjustments and fine-tuning, aiming to provide optimal results.
The ultimate goal is to develop a system for diabetes measurement using artificial intelligence. The input data set for this study consists of sugar water mixed with a pure water solvent, categorized into different types based on the sugar concentration. The project resulted in a system that achieved 82.0% accuracy in differentiating patient samples into diabetes positive and negative groups. The report document recorded the changes and adjustments made to produce these results, including the applied method, changes to calculation constants, operation diagrams, and library configurations that have demonstrated optimal accuracy for the input data set.
The findings aim to contribute to the field of diabetes early detection. Developing such a system requires significant dedication and resources, involving substantial investment in time, personnel, and equipment. Specifically, this extensive topic includes numerous steps, such as sample collection, sample screening, background determination, signal background correction, and the application of machine learning models. With a clear awareness of these complexities, the focus of this project is on processing the data before feeding it to machine learning and building a machine learning fusion system.
It explicitly excludes aspects such as measurement and data retrieval, equipment design, and certain processes. Keywords: Digital signal noise reduction, Raman signal processing, avoid fluorescence, Raman background interference, healthcare applications, machine learning, diabetes, precision medicine, non-invasive measures 5 Table of Contents THANKS. The necessity of the topic. Recent works on machine learning for Raman spectroscopy analysis.
Introduction of Artificial Intelligence. History of Artificial Intelligence in medicine. The future of AI in healthcare. Signal Processing in machine learning.
The introduction of signal processing. Benefits of preprocessing signals in machine learning. Introduction of Raman scattering. Application of Raman scattering in healthcare.
Background correction in Raman spectroscopy. Input Data for processing. Polynomial Fitting Method for baseline determination. Improved Modified Polynomial principles.2 Project set up.
Background correction results. Comparison of the signal before and after processing using SVM label classification. Accuracy of the unprocessed signals. Accuracy of the processed signals.
Adjustments and improvements. Using Extra Tree classifier instead of SVM for comparision. Potential clinical implications. Challenges for real-world application of the developed system.
57 7 List of Figures Figure 3.1 Workflow diagram of IMP fitting algorithm. (Jianhua Zhao, Harvey Lui, David I. MCLean, Haishan Zeng, 2007) .2 Built-in libraries for IMP Fitting in a launch file .3 Locate data file csv and extract unprocessed signals.4 Method to draw graph based on received signal.5 Method to save processed signals.6 Determine signals’ baselines and draw graphs.7 Determine the signal’s baseline with polynomial order equals 3.8 Determine the signal’s baseline with polynomial order equals 8.9 Determine the signal’s baseline with polynomial order equals 16.10 Attributions in definition of IMP in Pybaselines library.11 Background correction function.12 Background correction result with polynomial order equals 3.13 Background correction result with polynomial order equals 8.14 Background correction result with polynomial order equals 16.15 Function to limit the range of Wave number.16 Reorgazing Data form with 2 labels .17 Configuration for SVM.18 Accurary result check with SVM of unprocessed Raman signals .19 Accurary check with SVM of processed Raman signals with 8-polynomial order.20 Accurary check with SVM of processed Raman signals with 16-polynomial order.21 Accurary check with SVM of processed Raman signals with 12-polynomial order.22 Extra Tree configuration.23 Accurary check with Extra Tree of processed Raman signals with 12-polynomial order. The necessity of the topic Insulin is a vital hormone that helps regulate blood sugar levels and is crucial for maintaining the body's metabolic balance.
Diabetes, a long-term condition, occurs when the body is unable to effectively utilize insulin or does not produce enough of it, resulting in high blood glucose levels, a condition known as hyperglycemia, which can cause significant damage to various bodily systems, particularly blood vessels and neurons. Data from the Institute for Health Metrics and Evaluation reveals a significant increase in global diabetes cases and associated risks. Between 1980 and 2014, the number of people with diabetes increased from 108 million to 422 million, almost quadrupling over 35 years. This rise is more prominent in low- and middle-income countries, partly due to the growing prevalence of obesity and lack of physical activity.
As of 2014, statistics indicate that 8.5% of adults aged 18 and over have diabetes, directly causing about 1.5 million deaths, with nearly half occurring in individuals under 70 years old. Diabetes also contributes to 460,000 deaths from other kidney diseases, and roughly 1 in 5 deaths from cardiovascular disease can be attributed to diabetes. (Institute for Health Metrics and Evaluation, 2019) From 2000 to 2019, the global age-standardized mortality rate from diabetes saw a 3% increase, with a more significant 13% increase in lower-middle-income countries. In contrast, there was a 22% global decline in the likelihood of mortality from major non- communicable diseases between 2000 and 2019.
According to the International Diabetes Federation (IDF), as of 2021, an estimated 537 million people globally have diabetes, which equates to 1 in 10 adults aged 20–79 years. Furthermore, 1 in 6 babies born is affected by diabetes during fetal development, and up to 50% of adults with diabetes remain undiagnosed. (Institute for Health Metrics and Evaluation, 2019) Between 2000 and 2019, the global age-standardized mortality rate from diabetes saw a 3% increase. However, in lower-middle-income countries, the mortality rate from 9 diabetes surged by 13% during the same period.
In contrast, there was a significant 22% global decline in the likelihood of dying from any of the four major noncommunicable diseases (cancer, chronic respiratory diseases, diabetes, or cardiovascular diseases) between the ages of 30 and 70 from 2000 to 2019. According to the International Diabetes Federation (IDF), as of 2021, the world is home to an estimated 537 million individuals with diabetes, representing 1 in 10 adults aged 20–79 years. Shockingly, 1 in 6 babies born is affected by diabetes during fetal development, and up to 50% of adults have undiagnosed diabetes. (MINISTRY OF HEALTH, 2022) In Vietnam, nearly 5 million people are grappling with diabetes, with over 55% experiencing complications.
A survey by the Ministry of Health in 2021 estimates the adult diabetes incidence at 7.1%, affecting approximately 5 million individuals. Alarmingly, only around 35% of cases have been diagnosed, with an even lower percentage under management and treatment (23. (MINISTRY OF HEALTH, 2022) Projections from the IDF suggest a continued rapid increase in diabetes cases in Vietnam and globally. Recent works on machine learning for Raman spectroscopy analysis A lot of research on disease prevention methods relies on the combination of machine learning and Raman spectroscopy.
While there are variations in the studies regarding their approach, data samples, sampling conditions, machine learning techniques, and measurement tools, the use of machine learning in Raman spectroscopy-related methods undeniably yields positive results. This provides a solid foundation for further exploration into non-invasive disease diagnosis methods. In a research paper, “Recent Progresses in Machine Learning Assisted Raman Spectroscopy”, the authors demonstrated the potential of integrating machine learning techniques with Raman spectroscopy to enhance data analysis. The study involved an assessment of both traditional and modern statistical methods, including Principal Component Analysis, K-Nearest Neighbor, Random Forest, and Support Vector Machines, as well as deep learning algorithms like Artificial Neural Networks and Convolutional Neural Network.
The research showcased the broad applicability of 10 machine learning in fields such as materials science, biomedicine, and food science, leading to improved analytical accuracy and bulk identification. Additionally, the paper delves into the limitations of the study and outlines potential directions for future research. (Yaping Qi, Dan Hu, Yucheng Jiang, Zhenping Wu, Ming Zheng, Esther Xinyi Chen, Yong Liang, Mohammad A. Sadi, Kang Zhang, and Yong P.
Chen, 2023) Another research has shown that a combination of machine learning and Raman spectroscopy has proven effective in detecting and classifying breast cancer. This development is crucial, as breast cancer poses a significant health risk to women. Different subtypes of breast cancer react differently to drugs, which can significantly impact treatment outcomes. Therefore, accurately classifying these subtypes is of utmost importance.
The study applied Raman spectroscopy and machine learning techniques to streamline and expedite the process of distinguishing normal cells from breast cancer cells and classifying different subtypes of breast cancer. Raman spectra were collected from cultured breast cancer cell lines and analyzed using two machine learning algorithms: principal component analysis (PCA) - discriminant function analysis (DFA) and support vector machine PCA support (SVM). Both algorithms demonstrated an accuracy of over 97% in distinguishing normal breast cells from breast cancer cells, and over 92% accuracy in classifying breast cancer subtypes. The findings also support the use of characteristic Raman spectral features as biomarkers for cancer cells, such as the increased intensity of intrinsic Raman bands in cancer cells.