VIETNAM NATIONAL UNIVERSITY, HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY FACULTY OF COMPUTER ENGINEERING TRAN NGUYEN PHONG DAT DEO QUOC GRADUATION THESIS THE RESEARCH OF APPLYING FACE RECOGNITION FOR ATTENDANCE USING MOBILEFACENET MODEL XAY DUNG HE THONG DIEM DANH BANG NHAN DIEN KHUON MAT VOI MOBILEFACENET BACHELOR OF ENGINEERING IN COMPUTER ENGINEERING TP. HO CHI MINH, 2021 VIETNAM NATIONAL UNIVERSITY, HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY FACULTY OF COMPUTER ENGINEERING TRAN NGUYEN PHONG - 16521829 DAT DEO QUOC - 16521643 GRADUATION THESIS THE RESEARCH OF APPLYING FACE RECOGNITION FOR ATTENDANCE USING MOBILEFACENET MODEL XAY DUNG HE THONG DIEM DANH BANG NHAN DIEN KHUON MAT VOI MOBILEFACENET BACHELOR OF ENGINEERING IN COMPUTER ENGINEERING INSTRUCTORS NGHIA LE HOAI, M.SC DUY DOAN, PH. HO CHI MINH, 2021 PROTECTION COUNCIL OF THE GRADUATE THESIS Protection council of graduate thesis, established under decision. eee by Principle of University of Information Technology.
ACKNOWLEDGEMENTS First and foremost, we would like to express our sincere thanks to the faculty members of University of Information Technology in general and the Faculty of Computer Engineering in particular for imparting knowledge and valuable experience to us during their time at the school. Thanks to the guidance and instructions of the Faculty members, our graduation thesis is completed successfully. We would like to give our hight, respectful gratirude to our supervisor Mr. Nghia Le Hoai and Mr.
Duy Doan for their supervisor and support throughout this project. We have learned a lot from them. We would also like to express my eternal appreciation our family who has always been there for us, for all unconditional supports and patience. During the time at the university, we have received a lot of attention and help not only from our supervisor but also from our friends, seniors and officers who helped us to find information during the course.
We would like to thanks our dear friends and seniors. Finally, we really thanks to all people interested in helping us to finish this graduation thesis. In the process of making the thesis including this report, there still exist many surprises and shortcomings. We are looking forward to receiving your valuable comments and suggestions for our knowledge in this field.
Tran Nguyen Phong — Dat Deo Quoc Chapter 1. HH HH HH TH TT HT ng Hà 2 1. Goal and SCOD€. cv nền HH HH HH ciÕ 1.
DEFINITIONS AND HARDWARE OVERVIEW 2. Overview of the face human. Structure of face human. Emotion Of £aCO.
Face recognition OV€TVIW. cẶc St 2E HH H01 0 2. History of facial recognition technology. Pros ANd COINS.
¿+ S1 ST HH TH Hư 1 2. Basic definition of face reCOgnitÏOI.-- óc + vn TT TH HH dư 5 2. Algorithms for POJ€C. Ultra-Light face €f€CfOT.- «kg HH HH Hiếp 9 2.
Siamese neural netWOrk .- -- c1 +11 119 ng ng rệt 20 2. Outline of the alQorithMs. Embedded Computer and Hardware modules. Embedded Computer NVIDIA Jetson Tx2.
Embedded Computer NVIDIA Jetson Nano.- G0 12v T9 TH HH HH HH Hệ 31 Chapter 3. Methods of implemenfafIOI. Implementing OV€TVI€W .- - SH TH ng 33 3. Setting up Embedded Computer.
Using C270 camera for streaming live video to screen monitor. Preparing and training dataset. eee 5 1n ng giết 37 3. Recognizing face through video Sfream.
Technologies and tools. HH HH HH ng 42 1K na.- - -- c1 11H TH HH ng 44 Chapter 4. RESULTS AND EVALUATION. Test the first step: Face detection with Ultra-light algorithm.
Test the final step: Face recognition with MobileFacNet and FaceNet 42610177. CONCLUSION AND FUTURE WORK. 52 FIGURE MENU Figure 1-1: Overview of the face detection and recognition processes. 3 Figure 1-2: Draw each square where each face is detected.-- -----s«<-<-++ 4 Figure 1-3: Face Detection vs Facial Ñe€COBTIfIOH.- s5 51x ve eserreere 5 Figure 1-4: Steps of face recognition system applications .-- ------«<-<<++ 5 Figure 2-1: A man’s ÝAC€.-- t1 1 911911919109 HH HH Thọ nh nh ng 7 Figure 2-2: Identification points on the face of human.- -- 5+ s+s+<sss+sessss 9 Figure 2-3: Emotions in Í4C1aÌL.- s56 6E E31 91 911211 911 1111 HH ng 10 Figure 2-4: General pipeline of facial reCOETnIfIOHN.- 5 «5+ + *ss+eEeseese 13 Figure 2-5: Face recognition processing ÍÏOW.- ch gnrên 14 Figure 2-6: Automatic face €f€CfIOT.-- - c1 112139 9v vn ng ng ng rt 15 Figure 2-7: Face alignment example with the supervised descent method (SDM) ALQOTItHM 000.
16 Figure 2-8: Process of face f€DT€S€TfAfIOII.- --- 5ó c1 ng rưkt 17 Figure 2-9: Output of face recognition display on SCT€€H.-- 5-55 <£+s£<s+ 18 Figure 2-10: Siamese network. cọ TH ng HH 22 Figure 2-11: The goal of Ï€arnIn.- - - c1 3211133111111 1 1E 1E EErkkerere 23 Figure 2-12: Triple loss ẨunnCfIOH. c6 33213833 E*EEEEeEEEeeEEseeereeeerreeeereere 24 Figure 2-13: NVIDIA Jetson TX2 .-- ---- << 111 1 93 91 H1 HH ngư 27 Figure 2-14: Technical specifications of the Orbitty Carrier for Jetson TX2. 28 Figure 2-15: NVIDIA Jetson NanO.
Án HH HH HH ngư 29 Figure 2-16: Camera C270 Logitech. 32 Figure 3-1: Diagram of face recognition OVCTVICW. SG net 34 Figure 3-2: Jetson Tx2 connects to Input/Output deVIC€S.- 5555555 << 35 Figure 3-3: Testing Logitech camera with Ultra-Light face detector algorithm .37 Figure 3-4: Flowchart of training afaS€(.---- 5 s1 ng gưkt 38 Figure 3-5: Pretreatment to crop the image to standard size — before. 39 Figure 3-6: Pretreatment to crop the image to standard size — after.
-- 39 Figure 3-7: Flowchart of face recOgmition .:cescceescceseceneeeeeeeeseeceaeeceaeeesneeeneeesaes 40 Figure 3-8: OpenCV modular Structure .- -- 25 + S31 * + VE+EESeereseeeererereere 42 Figure 4-1: Face detection only one person iN a PICtUTe .- -- 55525 s<++++ss2 45 Figure 4-2: Face detection more than one person 1n a pICfUT€.-- -- «5+2 45 Figure 4-3: Wrong case of face đ@f€CfIOTI.-- G11 HH ng ng key 46 Figure 4-4: Test face detection in real-time .- --- 5 6 + + E*kEkkseeksrkrskesekree 46 Figure 4-5: One of result of test case with FaceNet model.-----««+««++ 49 Figure 4-6: One of result of test case with MobileFaceNet model .- 51 TABLE MENU Table 2.1: The correct recognition rates of appointed training.-- --- 5 + x13 91 3 vn nh ng ng, 26 Table 4.1: Result summary of performed tests with Jetson Tx2 KIt.2: Result summary of performed tests with Jetson Nano kIt. 50 ABREVIATION LIST AI - Artificial Intelligence AAM- Active Appearance Model DNN- Deep Neural Network ML - Machine Learning GPU - Graphics Processing Unit Open CV - Open Computer Vision PCA - Principal Component Analysis CUDA - Compute Unified Device Architecture CPU - Central Processing Unit SDM - Supervised Descent Method MTCNN - Multi-task Cascaded Convolutional Networks FERET- The Facial Recognition Technology REB - Receptive Field Block FP32 - Floating-point MFLOPs - Floating-point Operations Per Second LCD- Liquid-crystal display SDK - Software Development Kit ADM- Advanced Micro Devices ABSTRACT Nowadays, with the growing interest in computer vision, face detection and recognition has transcended from an esoteric to a popular area of research in computer vision and one of the better and successful applications of image analysis and algorithm based understanding, many applications implemented for face detection and recognition are used to achieve different types of projects, whether they are to be used for attendance systems in schools or for the check-in and check-out of employees in an organization. Because of general curiosity and interest in the matter, the authors research an attendance system using facial recognition. The purpose of this work is to research a system using face detection and recognition for attendance.
The system that is built in this project will be implemented on NVIDIA Jetson (Jetson Nano and Jetson Tx2). The system will include input video processing received from C270 Logitech camera sensor and input the human face recognition unit, then the results will be shown to the screen. Overview The human face is an essential part of the human perception system, and is used for practical applications such as surveillance, access control, and criminal identification. Face recognition is considered the most significant part in computer vision and image analysis, and thus it receives much more research in its different components such as the enhancement of its algorithms or new approaches created to better detect and organize the face.
Face recognition is a kind of biometric application for automatically identifying a person in front of a digital camera. It is essentially used in three major domains: the first use is in attendance systems and management of employee; the second is in visitor-notifying systems; and the third use is in access control systems. The recognition process 1s achieved by comparing the selected features from the face with previous facial information stored in the database. This method is perfect for some cultures that against fingerprint scanners because they don’t want to touch the same surface, especially in recent Covid-19 world situation that people have to do social distancing and should not touch the same surface.
With the knowledge of software and hardware, and the knowledge of an embedded systems engineer, our team will build a system for facial recognition with Ultra-Light face detect [1] and Mobile-Face-Net [2] model on NVIDA Jetson. For ease of visualization, the general process is illustrated in Figure 1-1. Figure 1-1 shows processing when the camera receives images of human faces, NVIDIA Jetson will process with facial recognition algorithm and display on screen the results of facial recognition. Input test: Person/Face/Image/Video Output: Id/Name will be displayed on the sereen a | Face =e 'TranNP' Jetson process with face recognition algorithms Camera connected with Nvidia Jetson Figure 1-1: Overview of the face detection and recognition processes 1.
Basic requirements The thesis is targeted to satisfy the following requirements: — Part 1: Face detection using Ultra-light face detection algorithm [1]: The main job in this section is to implement the algorithm on NVIDIA Jetson board. Specifically include the installation of the operating system for the board, install the necessary libraries. The face detection processing algorithm will proceed, draw each square where each face is detected on the live video monitor or picture, captured via Logitech C270 camera as illustrated in Figure 1-2. — Part 2: Face recognition using Mobile-Face-Net algorithm [2]: Preprocessing, we will prepare datasets for 10 people, each consisting of an mp4 video with video content showing the angles of the person's face (only 1 person).
Use face extraction algorithm to create a small image of human face (expected 112x112 pixels). Next step is the most important part, that is to implement Mobile-Face-Net algorithm [2] to compute, extract specificity of datasets in preprocessing section. Finally, identify the human face appearing in the live camera by displayed the name in the squares processed in part 1. Figure 1-3 shows the difference between face detection and recognition.
Facial Recognition Face Detection Figure 1-3: Face Detection vs Facial Recognition [3] Face Face Detection Recognition Figure 1-4: Steps of face recognition system applications 1. Goal and scope 1. Goal The goals of the research are: Research about Image Processing and Computer Vision. Study the face recognition technology.
Research about the basic of the theory, algorithm for detection and recognition face. Research about the NVIDIA Jetson Tx2, Jetson Nano [4] and camera module. Research about the Ultra-light and MTCNN [5] face detection algorithms. Research about recognizing face using Face-Net [6] and Mobile-Face-Net models.
Study about how to setup the development environment on Jetson Tx2 and Jetson Nano. — Use real-time image from the camera for recognition. — Build a real-time system on NVIDIA Jetson that can recognize people’s face for attendance check. This system will perform the operation in 3 main steps: e Detecting people’s face from pictures captured through camera.
e Recognizing faces within the picture and return the ID of each face as output. e The system will return a list of IDs that use for attendance check.