Vietnam National University, Ho Chi Minh City VNUHCM.-University of Information Technology Faculty of Computer Engineering Nguyễn Công Danh — 16520178 Nguyễn Đức Hoàng - 16520437 GRADUATE THESIS RESEARCH AND IMPLEMENTATION CNN ARCFACE ALGORITHM ON KIT ZYNQ7020 FOR FACE RECOGNITION ENGINEER OF COMPUTER ENGINEERING HO CHI MINH, 2021 Vietnam National University, Ho Chi Minh City VNUHCM.-University of Information Technology Faculty of Computer Engineering Nguyễn Công Danh — 16520178 Nguyễn Đức Hoàng - 16520437 GRADUATE THESIS RESEARCH AND IMPLEMENTATION CNN ARCFACE ALGORITHM ON KIT ZYNQ7020 FOR FACE RECOGNITION ENGINEER OF COMPUTER ENGINEERING MENTOR Ph. Nguyén Minh Son HO CHI MINH, 2021 PROTECTION COUNCIL OF THE GRADUATE THESIS Protection council of the graduate thesis, established under decision. by Principle of University of Information Technology A¬ e cece e ee eeee eee eee eeeeeneeeseeeeenees — President. Bo eee eee.
ACKNOWLEDGEMENT My group would like to express our sincere thanks to the teachers in the Faculty of Computer Engineering of the University of Information Technology, National University, Ho Chi Minh City, for creating opportunities for us to do the thesis In particular, we would like to give our sincere thanks and gratitude to our mentor, PhD Nguyen Minh Son. Thank you for supporting us with wholeheartedly during the time of our thesis. Sincere thanks to our friends and former students who helped us to learn more ideas and knowledge from previous topics. In the process of making the thesis, we have learned a lot more valuable experiences from reality and treasures techniques.
However, in the process of making it, mistakes can hardly be avoided, we hope that the council will forgive them. Thank you sincerely! Representative Student NGUYEN CONG DANH Faculty of Computer Engineering - 2021 CONTENTS CHAPTER 1. OVERVIEW OF THE PRO. The past and the future of face recognition.
The creation of face recognition. Presents and future predÏCfÏOIS. Research technology and related applications. Foreign Countries Research.
The problem that thesis focus on SOLVING .ssesssssssssssesssesssessesssesseeseeseesseesseessesseesaeenesaessaeesseeaeesneesseeneesas 5 72 We 0)622 ee 5 2. TRE T9 T10 EEE PETecenrovconusocurosssouscoesseecanesuesssvonsecousancosusovesenseest 8 2. Machine Learning in Face Recognition. Machine Learning Overview.
Machine Learning Models. Convolution Image PrOC€SSing. Arcface AlgorÏ(lhim.e«e«co«csseseeserseressrtsersrsrsrrserssrertsesaiirrssrriirrssre 18 PXŠ500 a2 76 11111. Transform Softmax to ATCÍACe.
Articles Evaluation ResuÌ(s. Vivado Design Suife. SYSTEM DESIGN AND IMPLEMENTATION. Vivado HLS ImplemenfafÏOn.
Project Implementaion PTOC€SS.-----e-cs«cseeeerseesrserrsrnsrsrsrree 51 CHAPTER 4. EXPERIMENTAL EVALUA TION. Linux Result Evaluation. Vivado Result Evaluation.
CONCLUSION AND FUTURE WORK. Future works and proposed TABLE OF FIGURES Figure 1: website paperwithcode.e‹sc-ccerteeereeerereiiiiriirirrrrseerrievẨÍ Figure 2: OpenCV ÏOgO.---ccccceceeeerrrrrtrtriiiirrirrriiiiiiiiirririiiiiiiirrrrirrriirarrrrrrrrrrrieeO Figure 3: DIib 100 w.scsssssssssssssssesssssssesesssesssesssssssssesesssssssesesssssesssessssessssessssessssessssssssssssssssssssssssssssssess7 Figure 4: Example of machine learning.ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssses LO Figure 5: Example of a neural network.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssees L2 Figure 6: Example of a training models.sssssssscsssssssseesesssssssssssuesssssssesssssussssssssssensee LO Figure 7: The example of small image and Kernel.sssssssssssssssssssssssssssssssssssssssssssssssssssssssses LA Figure 8: The formula of COnVOLUtION .sssssssssssssssssssssssssssssesssssssssesssssssssssssssssssssssssssssssssssssess LO Figure 9: Example of padding.----cssxxscieeeeereiiiiiiiiiiiiiirrriiirrrereeouee LO Figure 10: Example of the pooling layer.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssseees LO Figure 11: Example of Arcface algorithm .sesssssssssssssssssssesssssssssssssssssssssssssssssessssssssssssssssssss 20 Figure 12: Softmax aÏgOrithm.---ccvvcsvvevecrertkttttirrrrirtriiiiirrrrirrriiiiersreerrix 21 Figure 13: Weights Normalization. 21 Figure 14: Formula after adjustment. 22 Figure 15: Sphere Face LOSS.ssssssssssssssssssssssssssssssssssssssssssssssssssssssnsssssssssesssssssssssssssssssnsssssnsssssssss 2/2 Figure 16: Sphere Face Loss Piece WiS€ .ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssnsssssssssssnsssssss 2/2 Figure 17: Piece Wise FUnCtion.sssssssssssssssssssssssssssssssssssssssssessssssssssssssssssssssssssssssssssssssssnsssssssss 2D Figure 18: SphereFace-FNOrm .ssssssssssssssssssssssssssssssssssssssssssssssssssesssssssssesssssssssssssssnsssnsssssnsssnsssss 2D Figure 19: Additive Cosine Margin .sssssssssssssssssssssssssssssssssesssssssssesssssssssssssssssssssssssssssssssssssess 2E Figure 20: Additive Angular Margin LoS w.sssssssssssssssssssssssssssssssssssssssssesssssssssessssssssssssssssssss 2E Figure 21: Geometrical interpretation Of ATCÝaCe.-cc-cc-ccrrrrrreesereeeerrrrrre 2D Figure 22: Angle between the Feature and Target Center.ssssssssssssssssssssssssssssssssssssssses 20, Figure 23: Angle between the Feature and Target CenIeT.--e-eececccec 27 Figure 24: Decision margins of different loss function.
2Ÿ Figure 25: Angle distributions of both positive and negative pairs on different ataset .ssssssssecsessscssssssssesesessssssssssseessesssssssvesesesesssssssssesssesssessssssesesessssssssuueeseessssessssaueeessssssssssesessss OO) Figure 26: CMC and ROC curves of different models on MegaFace.2 Figure 27: ViVAO ÏOBO.--ccccceeeeeeerrerkrrtrrrririrrriiiiiiiriririiiiiiriirririririrrissssrrirrrrriooooe OO Figure 28: Example of Vivado IDEoeecssssssssesessssesesesesesesesesesssesesesesssssessseseseseseseessssesssssesesesees O7 Figure 29: Vivado HLS Design FloWsesssssssssssssssesesesssesssssesssssssssesssssssssssssssesesssssssssssssesseeeees 40) Figure 30: ZYNQ- 7000 SOC OVerVieW.eeeeeraeesruur 4A Figure 31: Z-turn Board (With ZYNÑQ-7020).cceeerrrssnsosooo 4O, Figure 32: Linux implementation system diagram.-------ieeesrsrrrrrerereeeee 4B Figure 33: Example of the biometrics numbers 49 Figure 34: Vivado HLS implemetation. 51 Figure 35: Implementation Process.csssssssssssssssssssssssssesssssesssesssesesesessseeeseseseseseeeeeeeseeeeessenseeeeeee OL TABLE OF TABLES Table 1: Verification performance (%) of open-sourced face recognition models on LFW, CALFW and CPLEW. 29 Table 2: Face identification and verification evaluation of different methods on Me gakFace. HH HH HH HH1 rttnrtrtrirtttktitrrorttirtttktiriorrrir OD LIST OF ACRONYMS ReLU Layers: Rectified Linear Units LFW: Labeled Faces in the Wild YTF: YouTube Faces CALFW: Cross-Age Labeled Faces in the Wild CPLFW: Cross-Pose Labeled Faces in the Wild HLS: High-Level Sunthesis PS: Processing System PL: Programmable Logic FFT: Fast Fourier Transform SRL: Shift Register LUT LUT: Look-up Table ABSTRACT Face recognition with the CNN algorithm is one of the most challenging problems nowadays.
Many programmers and scientists have created many algorithms and methods to solve this matter. These techniques compare to each other of how accurate and fast are they, and among all of those formulas, there is one stand out which called Arcface. Arcface is a new algorithm that was first to introduce in February 2019 in the paper titled Arface: Additive Angular Margin Loss for Deep Face Recognition and the researchers for this paper are four scientists: Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. This method is currently being research in many countries but the country that develops in China.
They have many papers and articles about this new algorithm. In our country, specifically Vietnam, at the moment this method doesn’t have much research and file or papers about this, most of the articles in Vietnam just about the introduction. The advantage of the proposed Arcface can be summarized as follows: Engaging, Effective, Easy and Efficient. But one of the difficulties in this method is that the paper and most of the example of this code is run on software, not many implement this algorithm down in the hardware system.
And because of that, our group is assigned to research this algorithm and trying to implement this Arcface method down to the hardware, specifically the kit Zynq7020 CHAPTER 1. OVERVIEW OE THE PROJECT 1. The past and the future of face recognition 1. The creation of face recognition In the early 1990s when the United States Department of Defense was looking for a solution or a technology that could spot criminals who furtively crossed borders.
And because of that, face recognition gained so much popularity. Then The Defense Department roped in eminent university scientists and experts in the field of face recognition for this purpose by providing them with research financing. In the early 2001, facial recognition made bold headlines after this technology was used in a public event which was at Super Bowl XXXV in Tampa by the law enforcement authorities to search for criminals or terrorists among the crowd. Soon after that, facial recognition systems were installed basically everywhere in the US in order to keep track of felonious activities.
Presents and future predictions Nowadays and maybe in the future, implement face recognition down into small devices or a kit is a trend in many parts of the world. By doing this, individuals can solve many problems in their daily life, for example they can create a simple face recognition devices for their houses, a smart home for short. Corporate can gain many benefits from this technology, because they can clear many simple problems such as check the attendance of an employee or strengthens their security measures. Research technology and related applications 1.
Domestic Research Our country — Vietnam, IC design or hardware design is not a major branch and because of that we have many limit in making or research in the field of face recognition. Our country doesn’t have enough expenditure, the technology is not reach to the requirement and the human resource is low, so that most of the face recognition research is come from development country. And Arcface is no exception, research papers about this algorithm in Vietnam are really little. The main reasons is that this method just got announced in 2019, and combine with the limited access to new technology, not much researchers in Vietnam learn and find out about this algorithm.
But in some programming community, people still talk about Arcface and have some demo about it. My prediction is that in the future, this technique will be widely use because of its speed and the high accuracy, and with all the things above, ourselves really proud of to be one of the first people to work with this algorithm. Foreign Countries Research Opposite with our country, other development country such as America, England, Japan, China,. have many research about face recognition.
There are so many papers about it but some got public, some remain as a confidential because of its potential and copy righted. According to the website paperswithcode.com, which is reliable website for programmers to study, and they even feature code and articles. The famous article about Arcface is the paper titled Arface: Additive Angular Margin Loss for Deep Face Recognition and the researchers for this paper are four scientists: Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. The paper include the introduction about Arcface, the math process about this algorithm and the comparision between this method and others.
This can be one of the most detailed article and might be the only article involved about this technique. Our group mainly use the article we have introduced above in order to create and execute our Figure 1: website paperwithcode. The problem that thesis focus on solving There are many problems that this dissertation can solved; however the specific things our group want to focus on is the implementation or the demo this algorithm on ARM side of the KIT. In the detailed syllabus many achievements have been introduced but doing this thesis, we cannot avoid drawbacks.
The biggest disadvantage our group has is that we do not acquire many articles; therefore we have changed our focus on the demo of ARM core. We believe that this way is one of many approachable directions about this technique. Also we have already successfully run the code on the Linux environment, and with the tool vivado, we can input our code C/C++ of Arcface into the tool to run the simulation or the synthesis. Therefore with many tools and many advantages, we decided to focus our thesis to this problem in order to prove that our vision and guess is right.
The Vivado tool provides us with man system in order to ease our simulation, such as synthesis function, analysis function and many more. With all of the above, we conclude that our main things we want to prove in this thesis is that the simulation or the focus on the ARM core of this algorithm. OpenCV OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.
Being a BSD-licensed product, OpenCV makes it easy for businesses to utilize and modify the code. The library has more than 2500 optimized algorithms, which includes a comprehensive set of both classic and state-of-the-art computer vision and machine learning algorithms. These algorithms can be used to detect and recognize faces, identify objects, classify human actions in videos, track camera movements, track moving objects, extract 3D models of objects, produce 3D point clouds from stereo cameras, stitch images together to produce a high resolution image of an entire scene, find similar images from an image database, remove red eyes from images taken using flash, follow eye movements, recognize scenery and establish markers to overlay it with augmented reality, etc. OpenCV has more than 47 thousand people of user community and estimated number of downloads exceeding 18 million.
The library is used extensively in companies, research groups and by governmental bodies. Along with well-established companies like Google, Yahoo, Microsoft, Intel, IBM, Sony, Honda, Toyota that employ the library, there are many startups such as Applied Minds, VideoSurf, and Zeitera, that make extensive use of OpenCV.