MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION GRADUATION PROJECT OF COMPUTER ENGINEERING TECHNOLOGY DESIGN AND IMPLEMENTATION OF A SMART ATTENDANCE SYSTEM APPLYING FACIAL RECOGNITION INSTRUCTOR: DR.LE MINH STUDENT: NGUYEN GIA HUNG NGUYEN DINH HONG QUAN SKL012541 Ho Chi Minh City, JANUARY, 2024 HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF INTERNATIONAL EDUCATION CAPSTONE PROJECT DESIGN AND IMPLEMENTATION OF A SMART ATTENDANCE SYSTEM APPLYING FACIAL RECOGNITION Students: NGUYỄN GIA HƯNG ID: 19119052 NGUYỄN ĐÌNH HỒNG QUÂN ID: 19119045 Major: COMPUTER ENGINEERING TECHNOLOGY Advisor: M. LÊ MINH Ho Chi Minh City, Jan 2024 THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- Ho Chi Minh City, January , 2024 GRADUATION PROJECT ASSIGNMENT Student name: _________________________ Student ID: ___________________ Student name: __________________________ Student ID: ___________________ Student name: __________________________ Student ID: ___________________ Major: ________________________________ Class: ________________________ Supervisor: ____________________________ Phone number: _________________ Date of assignment: _____________________ Date of submission: _____________ 1. Initial materials provided by supervisor: ___________________________________ 3. Content of the project: _________________________________________________ 4.
Final product: ________________________________________________________ CHAIR OF THE PROGRAM SUPERVISOR (Sign with full name) (Sign with full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- PRE-DEFENSE EVALUATION SHEET Student name:. Name of Examiner:. Content and workload of the project. Approval for oral defense? (Approved or denied) .) Ho Chi Minh City, Januray , 2024 EXAMINER (Sign with full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- EVALUATION SHEET OF DEFENSE COMMITTEE MEMBER Student name:.
Name of Defense Committee Member:. Content and workload of the project .) Ho Chi Minh City, January , 2024 COMMITTEE MEMBER (Sign with full name) DISCLAIMER The topic of this project had been researched and implemented seriously. We do not copy from any existing projects. All references have been fully cited.
We will be responsible for any violations that may occur. Authors (Sign and write full name) i ACKNOWLEDGEMENTS We would like to thank for M. Le Minh the enthusiastic help, as well as advice on the topic of this project, for useful information to complete the report. During the implementation process, although we have tried our best to complete the report, sometimes mistakes still occur.
Besides, we also want to thank the electronic component stores for providing us with good and required components to complete the report. Finally, we want to thank everyone who has supported our group's topic during the past time. ii ABSTRACT This report summarizes the process of ideation, research, design, construction, results achieved and development direction of the lighting control system. This system can be applied in practice so that the office can let employees automatically check in and check attendance when they come to the office.
The system mainly applies facial recognition and use hand gesture for analysis between check in and check out. With a database stored on google firebase, besides the Website, this system will make the work of management the information of employee more convenient and manager can monitor the total working day of each employee. This report can be used as a reference for students of Computer Engineering, Telecommunication Electronic Engineering, or those interested in recognition systems. iii TABLE OF CONTENTS LIST OF FIGURES.
vi LIST OF TABLES. viii LIST OF ABBREVIATIONS. ix Chapter 1: OVERVIEW. Scope of study.
7 Chapter 3: SYSTEM DESIGN. Central Processing Block. Power Supply Block. Connection diagram of the entire system.
Train model Yolo v8. Design UI interface for User. Web apps to manage data. Hand detection program.
Result of training Yolo v8 Model. Result of hand detection. Result of facial attendance. Result of Management website.
35 Chapter 5: CONCLUSIONS AND FUTURE WORK. 42 v LIST OF FIGURES Figure 2. Development time of YOLO network [1]. The structure of YOLO network [2].
Landmarks positions detected by Mediapipe [3]. Example of hand landmarks using in design. Logo of Google Firebase. Mobile app or Web app development process.
System’s block diagram. Logitech Webcam C270 HD. Raspberry Pi 4 Model B. System hardware diagram.
Structure of Yolo V8 [4]. Bounding box on Image User. Information about bounding boxes. The project’s interface frame.
The functions in mode. Main interface of management software. Flowchart of Attendance System. Flowchart of the function to automatically reset attendance time during the month and export to JSON data file.
Flowchart of Hand Detection. The results after training the model. Display application of Project. Show Information of the first employee.
Show Information of the second employee. Check IN and OUT successfully of first employee. Check IN and OUT successfully of second employee. Check already attended.
Add a new employee. Find employee function. Show information of first employee. Show information of second employee.
Delete Employee Function. Show information of all employee in the office. 38 vii LIST OF TABLES Table 3. Camera comparison table.
Mini computers comparison table. 13 viii LIST OF ABBREVIATIONS AI Artificial intelligence CNN Convolutional Neural Network CV Computer vision CPU Central Processing Unit CUDA Compute Unified Device Architecture CSV Comma Separated Values JSON JavaScript Object Notation YOLO You Only Look Once ix Chapter 1: OVERVIEW 1. Introduction Nowadays, with the rapid change of technologies and the development of Artificial intelligence, more and more smart appliances are created to meet humans needs. Activities performed by manual are gradually being replaced by Artificial Intelligent which is more convenient and more security.
Over the last few years, it has grown rapidly and been implemented widely in many fields, such as the facial recognition to unlock the smart phone or online payment, smart door system that allow people to open the door by facial recognition in smart homes or in modern car. To explore and learn about those fields, an idea about making a system that helps to check attendance in a small office has been made. With that idea, the group decided to implement the topic "Design and Implementation of a Smart Attendance System applying Facial Recognition”. The main goal of the system is to design and deploy a facial recognition-based attendance system for users.
This involves collecting data from the user's facial features to train a detection model. Additionally, users have the option to check in or out by using their fingers in front of the camera. The system will be fully automated, avoiding any physical buttons and relying solely on the camera to save costs. The system's data will be stored in a database (Firebase), allowing administrators to add or manage data either directly on Firebase or through the web application that we have developed.
Objectives The project "Design and Implementation of a Smart Attendance System applying Facial Recognition" was carried out to design a system with the following functions: - The system allows employee to check attendance automatically when they look at the camera of the system. The employee uses their finger to check in when they come to the office in the morning or check out the office in the afternoon. - The system has a screen for showing the information of the employee after they check attendance successfully. The system will show name, id, major and the address of the employee.
1 - The system also has a website for showing all the information of all employee in the small office, which has some functions such as add/delete employee, or find the information of the employee based on their ID number. - The data of the system will be sent to google firebase which is easily for storing and managing. Scope of study The scope of this project is based on the following tasks: - Design and implement an attendance system for employees in a decentralized office. - Utilize Raspberry Pi 4 as a signal and image processing unit within the system.
- Implement Yolo v8 model to process facial images of all employees based on a pre-customized dataset. - Deploy a database for the system on Google Firebase. - The system will operate based on the Internet, therefore, a stable internet connection is essential for seamless system functionality. Research methods The necessary research methods for this topic are as follows: - Find and learn about the Yolo V8 model - Conduct research using the OpenCV library and collect data for the dataset based on the Face Recognition module.
- Write a program to gather a dataset and integrate the Hand Detection module into this product. - Build and test the functions of the system. Outline The report is divided into 5 chapters: - Chapter 1: Overview 2 Introduces the project, its purpose, ways of research and the range for research of the project. - Chapter 2: Background Introduce some related topics, as well as compare them with the current topic.
- Chapter 3: System design From the requirements, this chapter will present the system block diagram, steps to design and implement for both the hardware system and software interface, as well as an algorithm for processing data. - Chapter 4: Result This chapter presents the result from the above design, check its functions and analyses the result. - Chapter 5: Conclusions and future work Based on the result, we will give conclusion about the advantages and disadvantages of the system, as well as some works for future development. YOLO Network The You Only Look Once (YOLO) method suggests employing an end-to-end neural network for simultaneous predictions of bounding boxes and class probabilities.
This contrasts with the conventional approach of repurposing classifiers in previous object detection algorithms to accomplish detection tasks. Development time of YOLO network [1]. The YOLO algorithm begins by taking an image as its input, employing a straightforward deep convolutional neural network to identify objects within the image. The underlying architecture of the CNN model integral to YOLO is illustrated in the diagram below.
The structure of YOLO network [2]. The initial 20 convolution layers of the model undergo pre-training with ImageNet, incorporating temporary average pooling and fully connected layers. Subsequently, the pre-trained model is adapted for detection, as prior research has demonstrated performance improvement by adding convolution and connected layers to a pre-trained 4 network. The ultimate fully connected layer in YOLO serves the dual purpose of predicting class probabilities and bounding box coordinates.
YOLO divides the input image into an S × S grid, designating each grid cell as responsible for detecting an object if the object's center falls within it. Within each grid cell, YOLO predicts B bounding boxes and assigns confidence scores to these boxes. These confidence scores gauge the model's certainty about the presence of an object within the box and the accuracy of the predicted box. During training, the objective is to assign one bounding box predictor to each object.
YOLO accomplishes this by designating the predictor with the highest Intersection over Union (IOU) with the ground truth as the "responsible" predictor. This specialization fosters improved forecasting for specific sizes, aspect ratios, or classes of objects, thereby enhancing the overall recall score. A pivotal technique employed in YOLO models is non-maximum suppression (NMS). NMS is a post-processing step that enhances the accuracy and efficiency of object detection.
Given that multiple bounding boxes may be generated for a single object, potentially overlapping or located at different positions, NMS identifies and removes redundant or incorrect boxes. The result is a streamlined output with a single bounding box representing each object in the image. Mediapipe Mediapipe is a library that is researched and developed by Google [3]. MediaPipe is a combination of a wide range of cross-platform machine learning solutions with several advantages: deployable on mobile, desktop, cloud, Web, IoT appliances, … it is open source and completely free (users can use and customize directly to fit their own problem).
Mediapipe supports almost all areas of Computer Vision. Some solutions include face detection, face mesh, hand detection, estimation human pose estimation, object detection, … and much more. Mediapipe also includes a hand detection solution to identify and locate hands.
