MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION GRADUATION THESIS INFORMATION TECHNOLOGY ANALYSIS OF THE HEALTHINESS OF FOOD THROUGH IMAGES LECTURER: TRẦN NHẬT QUANG, PhD. STUDENT: HO DANG TIEN LE HO HAI DUONG SKL012405 Ho Chi Minh City, December 2023 HO CHI MINH UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF INTERNATIONAL EDUCATION ---------- GRADUATION PROJECT ANALYSIS OF THE HEALTHINESS OF FOOD THROUGH IMAGES Students: HO DANG TIEN 19110059 LE HO HAI DUONG 19110073 Major: INFORMATION TECHNOLOGY Supervisor: TRẦN NHẬT QUANG, PhD. Ho Chi Minh city, December 2023 THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- Ho Chi Minh City, December 25, 2023 GRADUATION PROJECT ASSIGNMENT Student name: Hồ Đăng Tiên Student ID: 19110059 Student name: Lê Hồ Hải Dương Student ID: 19110073 Major: Information Technology Class: 19110CLA2, 19110CLA5 Supervisor: PhD. Trần Nhật Quang Phone number: 0378487371 Date of assignment: 11/09/2023 Date of submission: 25/12/2023 1.
Project title: ANALYSIS OF THE HEALTHINESS OF FOOD THROUGH IMAGES 2. Initial objectives provided by supervisor: Objective 1. Study food healthiness and its quantification methods. Apply machine learning to analyze food images: identify the type of food and evaluate its healthiness.
Content of the project Objective 1: Understand food nutrition. Understand food healthiness and its quantification methods. Build a food healthiness evaluation algorithm based on the chosen quantification method. Collect food nutrition data to calculate healthiness rating with the built algorithm.
Objective 2: Understand different deep learning concepts, models and implementation. Collect food images to build a dataset for models training. Build and evaluate the models to identify the type of food. Create user interface to implement the accomplishments of both objectives.
Final product: An analysis of healthiness of food through images system with user interface. CHAIR OF THE PROGRAM SUPERVISOR (Sign with full name) (Sign with full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- ---- SUPERVISOR’S EVALUATION SHEET Student name: Hồ Đăng Tiên Student ID: 19110059 Student name: Lê Hồ Hải Dương Student ID: 19110073 Major: Information Technology Project title: ANALYSIS OF THE HEALTHINESS OF FOOD THROUGH IMAGES Supervisor: PhD. Trần Nhật Quang EVALUATION 1. Content of the project:.
Approval for oral defense? (Approved or denied). Ho Chi Minh City, month day, year SUPERVISOR (Sign with full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- PRE-DEFENSE EVALUATION SHEET Student name: Hồ Đăng Tiên Student ID: 19110059 Student name: Lê Hồ Hải Dương Student ID: 19110073 Major: Information Technology Project title: ANALYSIS OF THE HEALTHINESS OF FOOD THROUGH IMAGES Name of Examiner:. Content and workload of the project. Approval for oral defense? (Approved or denied) .) Ho Chi Minh City, month day, year EXAMINER (Sign with full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- EVALUATION SHEET OF DEFENSE COMMITTEE MEMBER Student name: Hồ Đăng Tiên Student ID: 19110059 Student name: Lê Hồ Hải Dương Student ID: 19110073 Major: Information Technology Project title: ANALYSIS OF THE HEALTHINESS OF FOOD THROUGH IMAGES Name of Defense Committee Member:.
Content and workload of the project .) Ho Chi Minh City, month day, year COMMITTEE MEMBER (Sign with full name) ACKNOWLEDGEMENT First and foremost, with all of our uttermost appreciation, we would like to express gratitude towards our supervisor, Mr. Tran Nhat Quang. Throughout the progress of our capstone project, you have guided us with nothing but dedication and sincerity. You have provided us with a comprehensive perspective, which has helped me broaden my thinking to explore various interesting and innovative aspects that we have never taken into consideration before.
Thanks to that, we have come up with many new ideas which have been implemented to enrich the project. Moreover, you have imparted us practical wisdom accumulated during your life and career. As a consequence, we have learned many new fascinating things. Second, we are forever grateful to our families, who have been a solid foundation for us during the time we have been working on this project.
Finally, we want to show our gratitude to the teachers who have taught us in the previous courses. They have provided us the technical basis that we have been using in our study and work. These teachers are like mirrors reflecting successful lives, inspiring me to strive and hope for a bright future. We also want to thank the Information Technology department for providing the conditions for me and my peers to have a memorable learning experience.
Ho Chi Minh city, December 2023 Members of group: Ho Dang Tien 19110059 Le Ho Hai Duong 19110073 TABLE OF CONTENTS ABSTRACT. 2 CHAPTER 2: LITERATURE REVIEW AND THEORY BASIS. Impact of food [1]. Estimating impact of food choices on life expectancy: A modeling study [2].
The effect of presenting health and environmental impacts of food on consumption intention [3]. Importance of food labeling as a mean of information and traceability according to consumers [4]. The performance and potential of the Australian Health Star Rating System: a four-year review using the RE-AIM framework [5]. The Health Star Rating system – is its reductionist (nutrient) approach a benefit or risk for tackling dietary risk factors [6].
Consumers’ Perceptions of the Australian Health Star Rating Labelling Scheme [7]. Food classification using transfer learning technique [8]. Leveraging transfer learning to identify food categories [9]. 30VNFoods: A dataset for Vietnamese Foods recognition [10].
Transfer learning: a friendly introduction [11]. A Study on CNN Transfer Learning for Image Classification [12]. Health Rating System [13]. Health Star Rating algorithm.
Calculation of baseline points in the HSR system. Calculation of modifying points in the HSR system. Transfer learning with pre-trained model approach [22]. Loss Functions and evaluation metrics.
Categorical Cross-Entropy [23]. Fine-tuning in transfer learning. Base model accuracy and loss curves. Fine-tuned model accuracy and loss curves.
77 Table of Figures Figure 1. Categories of products in the HSR Calculator. ResNetV1 vs ResNetV2. ResNet50, ResNet101, ResNet152 layers.
Inception module with dimension reductions. Combination of inception and the residual module. Overfitting curve in training model. Image hash algorithms.
Image contains text added from outside. Image contains many different foods. Image containing the ingredients of the food. Image contains many smaller images combined.
Class distribution in the dataset. Categorical Cross-Entropy loss. Fine-tuning in transfer learning. Images after augmentation.
ResNet50V2 accuracy and loss curves. ResNet101V2 accuracy and loss curves. ResNet152V2 accuracy and loss curves. InceptionV3 accuracy and loss curves.
InceptionResNetV2 accuracy and loss curves. Xception accuracy and loss curves. Unfreeze all layers ResNet50V2. Fine-tuned ResNet50V2 accuracy and loss curves.
Fine-tuned ResNet101V2 accuracy and loss curves. Fine-tuned ResNet152V2 accuracy and loss curves. Fine-tuned InceptionV3 accuracy and loss curves. Fine-tuned InceptionResNetV2 accuracy and loss curves.
Fine-tuned Xception accuracy and loss curves. Predictions of ResNet50V2 on Test split. ResNet50V2's confusion matrix on Test split. Predictions of ResNet101V2 on Test split.
ResNet101V2's confusion matrix on Test split. Predictions of ResNet152V2 on Test split. ResNet152V2's confusion matrix on Test split. Predictions of InceptionV3 on Test split.
InceptionV3's confusion matrix on Test split. Predictions of InceptionResNetV2 on Test split. InceptionResNetV2's confusion matrix on Test split. Predictions of Xception on Test split.
Xception's confusion matrix on Test split .71 Table of tables Table 1. HSR baseline points for Category 1D, 2 and 2D. HSR baseline points for Category 3 and 3D. HSR baseline points for Category 1.
HSR V points for Categories 1D, 2, 2D, 3 and 3D. HSR V Points for Category 1. HSR Protein (P) and Fibre (F) points. HSR scores by category, with final Health Star Rating.
Keras model's performance on the ImageNet validation dataset. Model's layer unfreezes to. Ranking of models' performance on Test split. Ranking of models' performance on Validate split.
Health stars rated food data .77 Faculty for International Education – HCMC University of Technology and Education ABSTRACT Analysis of the healthiness of food through images is a complex problem which depends on food nutrition and image recognition model. This study presents a practical food healthiness analysis system through images that takes into account these factors through healthiness rating algorithm developed based on the Health Star Rating system. Nutritional values are collected through various sources and utilized as inputs for the healthiness rating algorithm. While images are labeled and pre-processed for transfer learning models to extract features and adapt through fine-tuning.
The trained models are then used to categorize images to identify the class of food, the system will then return the data from the healthiness rated food database accordingly. Keywords—transfer learning, analysis of the healthiness of food, food nutrition, Health Star Rating (HSR). 1 Chapter 1: Introduction CHAPTER 1: INTRODUCTION For the last decades, obesity has become a major health issue. According to World Health Organization (WHO), the worldwide prevalence of obesity nearly tripled between 1975 and 2016.
Once considered a high-income country problem, obesity is now on the rise in low- and middle-income countries, particularly in urban settings. Obesity is a major risk factor for non-communicable diseases, everyone should be aware of this, especially parents with children, since childhood obesity is associated with a higher chance of obesity, premature death and disability in adulthood. While, obesity is a persistent problem over the years, it is still preventable. WHO has stated that a supportive environments and communities are fundamental in shaping people’s choices, by making the choice of healthier foods and regular physical activity the easiest choice.
Having taken that statement that into consideration, we want to build an application that can provide the users the ease of access to food nutritional data on site or through images. There are main 2 objectives for our topic, after our team discussion. First, by studying how healthiness of food are rated from different organizations all around the world, we have created an algorithm to calculate the healthiness of food based on a few nutrition taken from food. Second, creating an image recognition model to categorize food so that the system can retrieve the correct food data from the system database.
After having finished those objectives, we decided to add a few more extra features to the application which are log and track the user daily food and beverages intake. On the first objective, we focus on understanding food nutrition, how each different healthiness rating algorithm works. Health Star Rating system developed by Australia and New Zealand Government was our most preferred choice. After reading the documentation on its application and guide to calculate the healthiness rating, we have developed our own algorithm based on it.
Furthermore, we also collect food nutritional value from various sources to feed into the algorithm. On the second objective, having to work with huge dataset of images taken from various sources around the internet, our team has taken advantage of the robust and efficiency of a deep learning technique, transfer learning. Transfer learning models including InceptionV3, InceptionResNetV2, ResNet50V2, ResNet101V2, ResNet152V2 and Xception are our top choices. These pretrained models were trained on more than 1 2 Chapter 1: Introduction million images distributed over 1000 categories range from animal, vehicles to plant, and most important of all for our topic, food.
These models have helped us a lot in building a food recognition model with good accuracy and low error. 3 Chapter 2: Literature review and theory basis CHAPTER 2: LITERATURE REVIEW AND THEORY BASIS 2. Impact of food [1] This article presents a brief but easy to understand the impact of food to human body as a whole. Food that we eat gives our bodies the nutrients they need to function properly; not getting the right nutrients, our metabolic processes suffer and our health declines.