MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING GRADUATION PROJECT COMPUTER ENGINEERING TECHNOLOGY TWO-LAYER SECURITY SYSTEM USING FACE RECOGNITION AND TIME ATTENDANCE INTEGRATION LECTURER: PHAM VAN KHOA STUDENT: DO MINH QUAN NGUYEN TRAN DUY KHANH SKL012539 Ho Chi Minh City, January 2024 HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF INTERNATIONAL EDUCATION GRADUATION PROJECT TWO-LAYER SECURITY SYSTEM USING FACE RECOGNITION AND TIME ATTENDANCE INTEGRATION DO MINH QUAN Student’s ID: 19119043 NGUYEN TRAN DUY KHANH Student’s ID: 19119063 Major: COMPUTER ENGINEERING TECHNOLOGY Supervisor: PHAM VAN KHOA, Ph.D Ho Chi Minh City, January 2024 THE SOCIALIST REPUBLIC OF VIETNAM Independence - Freedom - Happiness -------- Ho Chi Minh City, January 6th, 2024 PROJECT ASSIGNMENT Student name: DO MINH QUAN Student ID: 19119043 Student name: NGUYEN TRAN DUY KHANH Student ID: 19119063 Major: COMPUTER ENGINEERING TECHNOLOGY Class: 19119CLA1 Supervisor: PHAM VAN KHOA, Ph.D Phone number: Date of assignment: September 3rd, 2023 Date of submission: January 6th, 2024 1. Project title: TWO-LAYER SECURITY SYSTEM USING FACE RECOGNITION AND TIME ATTENDANCE INTEGRATION 2. Initial materials provided by the advisor: Documents such as papers about facial features extraction model - ArcFace. Content of the project: Analyze the challenge of the project, confirm the problem statement.
Learn about technical specifications, guiding through and theoretical basis of the components of hardware. Choosing model and summarizing the overall system. Design block diagram, flowchart, table of routing. Pre-processing data (clean, resize data, design schema for database) System configuration and design hardware Test run, check, debug, evaluate and adjust code.
Conduct report writing. Final product: A check-in check-out model that read RFID, extracts face’s data vector, a web application, a final report, a demo video. CHAIR OF THE PROGRAM SUPERVISOR (Sign with full name) (Sign with full name) i THE SOCIALIST REPUBLIC OF VIETNAM Independence - Freedom - Happiness -------- EVALUATION SHEET OF SUPERVISOR Student name: DO MINH QUAN Student ID: 19119043 Student name: NGUYEN TRAN DUY KHANH Student ID: 19119063 Major: COMPUTER ENGINEERING TECHNOLOGY Project title: TWO-LAYER SECURITY SYSTEM USING FACE RECOGNITION AND TIME ATTENDANCE INTEGRATION Supervisor: PHAM VAN KHOA, Ph. Content of the project: 2.
Approval for oral defense? (Approved or denied) 7. Mark: - in words: Ho Chi Minh City, January , 2024 Supervisor (Sign with full name) ii THE SOCIALIST REPUBLIC OF VIETNAM Independence - Freedom - Happiness -------- PRE-DEFENSE EVALUATION SHEET Student name: DO MINH QUAN Student ID: 19119043 Student name: NGUYEN TRAN DUY KHANH Student ID: 19119063 Major: COMPUTER ENGINEERING TECHNOLOGY Project title: TWO-LAYER SECURITY SYSTEM USING FACE RECOGNITION AND TIME ATTENDANCE INTEGRATION Examiner: EVALUATION 1. Content of the project: 2. Approval for oral defense? (Approved or denied) 5.
Mark: - in words: Ho Chi Minh City, January , 2024 EXAMINER (Sign with full name) iii THE SOCIALIST REPUBLIC OF VIETNAM Independence - Freedom - Happiness -------- EVALUATION SHEET OF DEFENSE COMMITTEE MEMBER Student name: DO MINH QUAN Student ID: 19119043 Student name: NGUYEN TRAN DUY KHANH Student ID: 19119063 Major: COMPUTER ENGINEERING TECHNOLOGY Project title: TWO-LAYER SECURITY SYSTEM USING FACE RECOGNITION AND TIME ATTENDANCE INTEGRATION Name of Defense Committee Member: EVALUATION 1. Content of the project: 2. Mark: - in words: Ho Chi Minh City, January , 2024 COMMITTEE MEMBER (Sign with full name) iv DISCLAIMER We hereby confirm that this project is the result of our independent research conducted under the guidance of Dr. Pham Van Khoa.
The statements and discoveries presented herein are the outcome of our meticulous and independent exploration, involving comprehensive examination and analysis of academic materials. Researchers accountable for this project, we commit to refrain from reproducing or duplicating the content and conclusions of other works. All references used have been appropriately acknowledged and referenced in their entirety. All information, data, and research findings presented in this project are for reference purposes only and do not represent a direct affiliation or association with any specific organization, institution, or individual.
This research has been carried out solely for academic purposes and not for commercial or personal gain. The author and research team do not assume responsibility for any consequences or applications arising from the utilization of information in this project without validation or support from experts in the relevant field. Any implications or use of information from this project are entirely at the discretion and responsibility of the reader and are independent of the author or supervisor. STUDENTS (Sign with full name) v ACKNOWLEDGEMENTS We extend our heartfelt appreciation to numerous individuals and institutions who have been instrumental in our academic journey.
Foremost, our deepest gratitude goes to Dr. Pham Van Khoa, our advisor, for his unwavering support, patience, insightful guidance, and invaluable contributions to this thesis. His enthusiastic mentorship, wealth of knowledge, and expert advice in artificial intelligence were instrumental in the successful completion of this research. Without his encouragement and direction, this endeavor would not have come to fruition.
We also wish to express our sincere gratitude to Ho Chi Minh City University of Technology and Education for fostering a dynamic and enriching learning environment, providing opportunities through contests and seminars. Additionally, our heartfelt thanks go to the dedicated teachers who have imparted invaluable knowledge and wisdom during our four-year journey. Their teachings have not only enriched our academic pursuits but have also honed our practical skills and experiences. We are indebted to our seniors and friends for their steadfast support and invaluable assistance throughout this academic endeavor.
Their encouragement and collaborative efforts have been immensely beneficial. vi TABLE OF CONTENTS PROJECT ASSIGNMENT. i EVALUATION SHEET OF SUPERVISOR. ii PRE-DEFENSE EVALUATION SHEET.
iii EVALUATION SHEET OF DEFENSE COMMITTEE MEMBER. vi TABLE OF CONTENTS. vii LIST OF FIGURES. ix LIST OF TABLES.
xi LIST OF ABBREVIATIONS. ARCFACE AND MOBILENETV2 INTEGRATION. RASPBERRY PI 4 MODEL B. RADIO FREQUENCY IDENTIFICATION.
RTC INTEGRATED CIRCUIT. REQUIREMENTS AND BLOCK DIAGRAM OF SYSTEM. REQUIREMENTS OF SYSTEM. BLOCK DIAGRAM OF SYSTEM.
CENTRAL PROCESSING BLOCK. GET DATA AND SEND DATA PROGRAMS. FRONT-END/CLIENT LAYER. MISSIONS OF WEB APPLICATION.
COMPRESSED FACIAL FEATURES EXTRACTION MODEL. OVERVIEW OF MODEL. TESTING MODEL ABILITY. WEB APPLICATION INTERFACE RESULTS.
FACIAL FEATURES EXTRACTION MODEL RESULTS. SIMILARITY OF COMPRESSED AND ORIGINAL MODELS. EXTRACTION OF COMPRESSED MODEL. COMPLETE HARDWARE WITH ENCLOSURE.
CONCLUSIONS AND RECOMMENDATIONS. 83 viii LIST OF FIGURES Figure 2. The MERN Stack work methodology. Training flow supervised by ArcFace loss.
The difference between residual block (a) and inverted residual block (b). Map of Raspberry Pi 4 Model B. GPIO of Raspberry Pi 4 Model B. SPI communication protocol.
Schematic circuit of NOTIFICATION block. Schematic circuit of LOCK CONTROL block. Schematic circuit of hardware. Printed circuit board of hardware.
Implemented circuit board of hardware. Flowchart of GET DATA program. Flowchart of SEND DATA program. Flowchart of ATTENDACE program.
Flowchart of FACE ADD sub-program. Flowchart of FACE ADD ADD sub-program. Flowchart of PIN INPUT sub-program. Flowchart of RFID INPUT sub-program.
Flowchart of FACE SCAN sub-program. Flowchart of FACE CHECK sub-program. Flowchart of ACCESS HANDLING sub-program. Web application’s architecture.
User and admin use case interface. Admin routes control. Hardware request routes control. Clock information flowchart.
User routes control. System database design in MongoDB. Workflow of feature extraction model. Output facial feature extraction files.
Output precision of model. Sign in interface. Main dasboard interface. Clock Information Page.
Manage Users Page. Detail Users Information Page. Update Information for User. Create New User page.
Distribution of vectors - orginal model on Colab. Distribution of vectors - compressed model on Colab. Statistics of vectors - orginal model on Colab. Statistics of vector - compressed model on Colab.
Distribution of vectors - compressed model on Raspberry. Statistics of vectors - compressed model on Raspberry. Result of feature extraction verification. RFID - FACE ADD window.
FACE ADD - facial extraction window. RFID - LEVEL 1 window. PIN - LEVEL 1 window. PIN INPUT - LEVEL 1 window.
FACE SCAN - LEVEL 2 window. FACE SCAN - facial extraction window. Complete hardware with enclosure (1). Complete hardware with enclosure (2).
77 x LIST OF TABLES Table 3. Connect pins between Raspberry Pi 4 and DS1307. Connect pins between Raspberry Pi 4 and RC522. Typical current and typycal voltage of hardware.
Hardware components list. Admin endpoint table. Hardware request endpoint table. User endpoint table.
Dataset for tranining pre-trained model. Comparison person1 scan vector with person1 database. Comparison person1 scan vector with person2 database. Comparison person1 scan vector with person3 database.
Comparison person1 scan vector with person4 database. 75 xi LIST OF ABBREVIATIONS ABBREVIATIONS MEANING 1 API Application Programming Interfaces 2 ARM Advanced RISC Machine 3 ASCII American Standard Code for Information Interchange 4 BSON Binary JavaScript Object Notation 5 CNN Convolutional Neural Network 6 CSS Cascading Style Sheets 7 DSI Display Serial Interface 8 GPIO General Purpose Input/Output 9 GUI Graphical User Interface 10 HDMI High-Definition Multimedia Interface 11 HMAC Hash-based Message Authentication Code 12 HTML HyperText Markup Language 13 HTTP Hypertext Transfer Protocol 14 I2C Inter-Integrated Circuit 15 ID Identification 16 IoT Internet of Things 17 IPS In-Plane Switching 18 IT Information Technology 19 JSON JavaScript Object Notation 20 JWT JavaScript Object Notation Web Token 21 KDF Key Derivation Function 22 LCD Liquid Crystal Display 23 LED Light Emitting Diode 24 LPDDR4 Low Power Double Data Rate 4 25 MEAN MongoDB, Express, Angular, Node.js 26 MERN MongoDB, Express, React, Node.js 27 MISO Master In Slave Out 28 MOSI Master Out Slave In 29 MVC Model View Controller 30 PC Personal Computer 31 PIN Personal Identification Number 32 RDBMS Relational Database Management System xii ABBREVIATIONS MEANING 33 RFID Radio-Frequency Identification 34 RGB Red, Green, Blue 35 RTC Real-Time Clock 36 RX Receive 37 SCK Serial Clock 38 SCL Serial Clock 39 SDA Serial Data 40 SDRAM Synchronous Dynamic Random-Access Memory 41 SHA-1 Secure Hash Algorithm 1 42 SPI Serial Peripheral Interface 43 SS Slave Select 44 TFLite TensorFlow Lite 45 TOTP Time-Based One-Time Password 46 TV Television 47 TX Transmit 48 UART Universal Asynchronous Receiver/Transmitter 49 UI User Interface 50 UK United Kingdom 51 USB Universal Serial Bus xiii ABSTRACT In today’s rapidly evolving business landscape, the escalating demand for robust data storage services mirrors the expansive growth of companies across diverse domains. Within this context, the effective management of employee working hours has emerged as a critical aspect, exerting a profound influence on company finances. Despite its pivotal role, many enterprises, ranging from small businesses to some larger counterparts, lack stringent enforcement of attendance tracking, relying on single-layer security measures that inadvertently expose them to unauthorized access and potential financial losses.
To address this challenge, this paper introduces a sophisticated two-layer security system that seamlessly integrates embedded computing and artificial intelligence. This innovative solution aims to rectify the vulnerabilities associated with conventional attendance tracking systems, particularly those susceptible to unauthorized access and fraudulent practices such as proxy attendance. The proposed system includes a web application, providing users with a comprehensive platform not only to monitor their attendance data but also to access the first layer of security through a credential pin code. Users have the flexibility to choose between an access card and a pin code for the first layer, ensuring that access information is uniquely tied to individual employees.
Additionally, the system incorporates an advanced artificial intelligence model to enhance the second layer of security, integrating facial recognition technology. This multi-layered approach significantly diminishes the likelihood of attendance manipulation and unauthorized access, thereby safeguarding the integrity of employee working hours and, consequently, the financial interests of the company.