VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY INFORMATION SYSTEM FACULTY PHI LONG NGUYEN INFORMATION SYSTEM ENGINEERING Ho Chi Minh City, 2022 VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY UNIVERSITY OE INFORMATION TECHNOLOGY INFORMATION SYSTEM FACULTY PHI LONG NGUYEN - 18521043 GRADUATION THESIS BUILDING AN ANDROID APPLICATION FOR FISH RECOGNITION INFORMATION SYSTEM ENGINEERING ADVISOR Dr. PHAN XUAN THIEN Ho Chi Minh City, 2022 INFORMATION OF THE GRADUATE THESIS COUNCIL œ£lw› Graduation thesis grading committee, established under Decision No. of the Rector of the University of Information Technology. — Commissioner ACKNOWLEDGE calle For the purposes of completing this graduation thesis, besides my own efforts and con- stant efforts, it is impossible not to mention the support and help of the teachers work- ing at the University of Information Technology, VNU-HCMC.
I would like to express my deep and sincere thanks to Dr. Phan Xuan Thien, my instructor. He has whole- heartedly helped me since the days I started to study deep learning and has trusted and encouraged me during difficult times while working on this thesis. In addition, I am incredibly grateful to him for contributing ideas from the days of applying for the topic.
His guidance helped me in all the time of research and writing of this thesis. I could not have imagined having a better advisor and mentor for my thesis. In the process of implementation, despite efforts to learn, research, experiment and initially achieve some encouraging results, but due to limited knowledge and expe- rience, it is inevitable that shortcoming; I look forward to receiving your comments to edit and improve the thesis. Ho Chi Minh City,.
Advisor Signature Chapter 1: Overview 1. Goals and Study SCOpe. (th 1EHư 5 Chapter 2: Methods. Artificial Neural NetWOTK.
Convolutional Neural Ne€tWOrK.- «+6 +5 xxx SH gi, 9 2. -- + S523 12EE1 1E kg He, 11 2. The Input ÏDafa. - Sàn E11 TH HH HH 12 2.
sUMMEREMEEETT ss secsesecseceasessssneseeuseessessssesceseseasesseneeseseeee 25 P000. Sub-sampling or Pooling LLAY€T. vn HH HH HH HH Hit 28 2. Fully connected ÌLAy€T.
-- + tk ng Hye 29 na. Pre-trained ResNet Model. CCaAÀẠỤDIIAỤỘỤDŨDỤDẮẠŨẲŨỘỤŨỖỒ. cece cece es ceeeseseseseeseseseecsnesessseesensneeeseeeees 44 2.- ‹- tt ng He 50 2.- ¿cv HH HH ng HH Hit 51 Chapter 3: Android Application.
Retrofit Chapter 4: EVaÌuatiOn.-- «+ St kề 1E T121 11011111 0H11 TH già. Sp irre: IR coe ecessssensesssceveoneeesnenusseoneeusseaneessneness 59 4. Sia os đợt. MOO so scsseveseevesnsseavesnscesnsusoesnsceseenueesanense 61 4.
5 SE tk LH 2211101100101 Ho HH1 0T go re 61 4. User Ïn(€TÍAC€. 1h HH TH HH HH HH HH HH HH Hàn 62 4. - «1h 11T TH HT HH 66 5.
Future Direction References. Vietnamese References List of Figures Figure 1.1 Seafood Production wild fish catch vs aquaculture from 1960 to 2010 Figure 2.1 Feed-forward Neural Network Figure 2.2 Artificial Neural Network Architecture Figure 2.3 ANN Perceptron Figure 2.4 Convolutional Neural Network Architecture Figure 2.6 Array of Pixel in Digital image.7 Grayscale Image and RGB Color Image.9 Convolution Layer Operation.10 How Convolution Layer Work .11 Convolutional Layer Figure 2.12 Convolution Operation Problem Figure 2.13 The Input Matrix with Padding p=1 Figure 2.14 Feature Map with Padding Applied.15 Input Matrix with Stride of One.16 Feature map with Stride applied .17 Reject region when kernel goes out of the matrix with s = 2 Figure 2.18 CNN for Car Recognizing.19 Linear Function and Non-linear Function .20 Linear Function Problem Figure 2.21 Non-linear function.22 Activation Functions Figure 2.23 ReLU Function Figure 2.24 ReLU Activation Mapped on Feature Map.25 Difference between ReLU and Leaky ReLU.26 Pooling Example Figure 2.27 Max Pooling Example .28 Average Pooling Example.29 CNN Overview Figure 2.33 Knowledge Transfer Example .34 Transfer Learning Architecture 33 Figure 2.35 VGG16 Pre-Trained Model after Fine-tuning .36 Training error and Test error on CIFAR-10 with 20-layer and 56-layer.37 A residual block - the fundamental building block of residual networks.38 Example network architectures for ImageÌNet.39 Training curves on ImageNet-1K 37 Figure 2.40 A Large-Scale Fish Dataset from Kaggle.41 Fish Species Dataset from MendelayData.42 A Number of Images Are Used In Thesis .43 Some Species Samples of Dataset Al Figure 2.44 Deep Learning Process 42 Figure 2.45 Input Images After Preprocessing 43 Figure 2.46 The Input Image Under Machine’s Perspective Ö44 Figure 2.49 ResNet Architecture In Our Model.50 Feature Map After Blocks Applied.51 Deep Learning Frameworks Adoption 2021 .52 One-Hot Encoder.54 Model Accuracy Per Epoch 52 Figure 2.56 Evaluation on Test Dataset Result 53 Figure 3.1 TF Model To TFLite 54 Figure 3.3 System Architecture 56 Figure 3.1 Our Project Restful API.3 Main screen 62 Figure 4.5 Two options for importing image 64 Figure 4.6 Image Scan Result 65 Figure 4.7 Camera Scan Result .67 List of Tables Table 2.1 Comparison between ResNet-50 and ResNet-101 on ImageNet-IK.2 Model Compile Configuration Table 2.3 Model Training Configuration .1 Pre-trained Models Details Table 4.2 Pre-trained Models Training and Testing Results. List of Acronyms AI: Artificial Intelligence API : Application Programming Interface ANN: Artificial Neural Network CNN: Convolutional Neural Network MLP : Multilayer Perceptrons NN: Neural Network FC Layer: Full-connected Layer RNN : Recurrent Neural Network ReLU: Rectified Linear Unit ResNet : Residual Network Abstract The high demand for fishing nowadays causes the risk of overfishing which can lead to the depletion of aquatic resources, especially river and sea fishes. In addition, the lack of information about the fish species and further details of fishes may also affect the productivity and effectiveness of fishing activities.
Due to that fact, we believe that building a tool that can recognize fishes automatically and supply detailed information about fishes to improve the productivity of fishing and help the government and organizations control fishing activities more efficiently is ur- gently necessary and important, especially in developing countries like Vietnam and some other countries. Toward that goal, we research and develop method that can au- tomatically detect and recognize fish species based on photos or scanning images of them. In our proposal, a Convolutional Neural Network for classification, and transfer learning is also applied to improve the accuracy of the classification. Our experiments show that the accuracy after validating is 89.13%, which is an acceptable and promis- ing result.
Moreover, to enhance the applicability of our proposal, we also build up a software tool for Android mobile devices based on our method to help users approach and use our method easily and efficiently. Keywords: Convolutional Neural Network, CNN, Deep Learning, Machine Learning, Neural Network, Fish Classifications, ResNet, Keras. Chapter 1: Overview The content of this chapter presents the problem statement, an overview of the problem, the challenges encountered, the goal - scope of the thesis and finally the lay- out of the thesis. Problem Statement According to 2022 State of World Fisheries and Aquaculture (FAO, 2022), production of aquatic animals in 2020 was more than 60 percent higher than the average in the 1990s, significantly outpacing world population growth, largely due to increasing aq- uaculture production and fishing.
Global consumption of aquatic foods increased at an average annual rate of 3.0 percent from 1961 to 2019, a rate almost twice that of annual world population growth (1.6 percent) for the same period. Most countries are easy to see a rise in their aquatic food consumption per capita at that time. Therefore, in order to satisfy the demand of seafood supply, the rate of overfishing will increase. Some of the reasons for this problem are listed below: 1.
Lack of knowledge regarding fish populations. Difficulties in regulating fishing areas due to lack of resources and tracking activity. Fishing areas are largely unprotected. To be more detailed, owing to the lack of knowledge, it will lead to the situa- tion, overfishing, that people will catch even small fish that are not within the allowed size.
In addition, the insufficiency of information can cause people to catch and kill ra- re and precious fish species that need to be protected to maintain the ecosystem. Seafood production: wild fish catch vs aquaculture, World Crier Aquaculture is the farming of aquatic organisms including fish, molluscs, crustaceans and aquatic plants. Capture fishery production is the volume of wild fish catches landed for all commercial, industrial, recreational and subsistence purposes. Capture fisheries 80 million t Aquaculture 60 million t 40 million t 20 million t 0t 1960 1970 1980 1990 2000 2010 Source: Food and Agriculture Organization of the United Nations (via World Bank) OurWorldInData.org/fish-and-overfishing * CC BY Figure 1.1 Seafood Production wild fish catch vs aquaculture from 1960 to 2010 In Figure 1.1, the fish catch, and aquaculture data published by Hannah Ritchie and Max Roser.
Globally, the share of fish stocks which are overexploited — we catch them faster than they can reproduce to sustain population levels — has more than dou- bled since the 1980s. Problem Solution Due to that fact, we believe that building tool that can recognize fishes automatically and supply detailed information about fishes to improve productivity of fishing and help government and organizations controlling the fishing activities more efficiently is urgently necessary and important, especially in developing countries like Vietnam and some other countries. Toward that goal, we research and develop method that can au- LH. Roser, Our World in Data, 2021.org/fish-and-overfishing tomatically detect and recognize fish species based on photos or scanning images of them.
In our proposal, a CNNs model used for classification, and transfer learning is al- so applied to improve the accuracy of the classification. More specifically, we used the pre-trained model ResNet101- a CNN with 101 layers deep. ResNet-101 is an im- proved model from the ResNet-50 and ResNet-50V2 versions. Our experiments show that the accuracy after validating is 89.13%, which is an acceptable and promising re- sult.
Moreover, to enhance the applicability our proposal, we also build up a software tool for Android mobile devices based on our method to help users approach and use our method easily and efficiently. Challenges Since we will have to recognize fishes through existing photos or new ones taken with the camera, the image quality will not be as good as with natural environmental condi- tions. In natural environments, any classification task is challenged by diversity in background complexity, turbidity, and light propagation will all reduce the accuracy of deep learning. Furthermore, we want to be easy, convenient, and accessible for everyone, so we will have to bring deep learning to electronic devices, specifically in this thesis, Android phones.
This is also a challenge in identifying fishes using CNN algorithms in deep learning. On the current phone models, no equipment can compare with a com- puter so running the deep learning process, which can be up to several hundred, even thousands of tasks and algorithms are something that is not easy for data science. Goals and Study Scope Before starting the thesis, we need to clarify the goal of the research direction that we aim to. To help people know more information and fish individuals (e., scientific name, size, as well as habitat).
We have studied image classification, thereby helping readers to understand more clearly about classification algorithms, how computers learn to perform tasks. With the scope of the graduation thesis, the main objectives of this thesis will include: 1. Research on an overview of the problem of object recognition in images. Learn about the architectures in the deep learning recognition model.
Build models and apply on Android phones. Conclusions and future development orientations. Thesis Structure The thesis is organized into five chapters, the main content of each chapter is as fol- lows: Chapter I: Overview: An overview of the content of the thesis topic, including: posing problems, solutions, challenges encountered in the process and finally the goals and scope of the research. Chapter 2: Methods: In this chapter, the thesis will present about the types of neural networks, their architecture as well as how they work.
In addition, description of the data set and experimental process, model building. Chapter 3: Android Application: This chapter goes into detail about the Android application that the trained model applied.