NATIONAL UNIVERSITY OF HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY FACULTY OF COMPUTER ENGINEERING VU MINH HOANG NGUYEN KHANH MINH TAN GRADUATION THESIS Smart Pulpit with IoT for smart classroom Buc giáng thông mình trên nền tang IoT ứng dụng trong lớp hoc thông minh ENGINEER IN COMPUTER ENGINEERING HO CHI MINH CITY, 2021 NATIONAL UNIVERSITY OF HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY FACULTY OF COMPUTER ENGINEERING VU MINH HOANG - 17520523 NGUYEN KHANH MINH TAN - 17521015 GRADUATION THESIS Smart Pulpit with IoT for smart classroom Buc giảng thông mình trên nền tang IoT ứng dụng trong lớp học thông minh ENGINEER IN COMPUTER ENGINEERING INSTRUCTOR PhD. NGUYEN MINH SON HO CHI MINH CITY, 2021 INFORMATION OF THE GRADUATION COURSE GRADING BOARD The board marks the graduation course, established under Decision No. of the President of the University of Information Technology. THANK YOU To successfully implement the graduation thesis section with the topic “Smart Pulpit with IoT for smart classroom “in addition to the efforts of the group - we always receive dedicated help from teachers, colleagues, friends, and many agencies and organizations.
First, we would like to express our sincere thanks to the University of Information Technology — VNU Ho Chi Minh City, the board of directions, and teachers at the Faculty of Computer Engineering for creating the best conditions for us to have a dynamic and effective learning environment throughout the learning process at the school. Thanks to the guidance of our teachers, we have learned and accumulated the foundational knowledge of Computer Engineering. We especially thank Ph. Nguyen Minh Son and PhD.
Do Tri Nhut for always supporting and guided us throughout the implementation of the topic. We learned many useful and necessary knowledge, as well as having the opportunity to practice and cultivate the above knowledge. Finally, we sincerely thank our families, parents, who are always there to support us, help us have time to study the topic and wholeheartedly support us. During graduation, despite trying to complete the topic, it is inevitable that there are some faults.
We are very welcome and sincerely thank the comments of teachers and readers for the thesis to be improved. We hope this thesis will be a useful reference for those who are interested in relevant areas of the subject. Ho Chi Minh City, July 2021 Student Vu Minh Hoang Student Nguyen Khanh Minh Tan Sincerely thank you! TABLE OF CONTENTS Chapter 1. Smart classfOOm OVeTVi€W.
What is Smart ClassroOm?. Benefits of smart ClassrOOms. Benefits for teachers. Benefits for Student.
What is a smart pulpit? cece LO 2. Teacher Recognition SyS(em. Aniti-fraud SYStM. Lecture presentation system.
Software Architecture ws OD 3.eeeierrrrririiirrririirrrrruu OO Chapter 4. IMPLEMENTATION AND EXPERIMENTAL,. Auto attendance serversD2 4. Experimental conc]uSÏOH.---- ccccccsstccsevexvvseeerietrirtkettrrrrrirrririirsrrioe DD Chapter 5.
CONCLUSION AND ORIENTED DEVELOPMENT. PICTURE CATEGORY Figure 2. 1 Picture about Smart class of ANU_V[12] ,. 2 Smart class of Blingnetworks[ 13] ssscccssssssssssssssssssssssssssssssssssssssssssssssssssssssssssesstete Figure 2.
3 Basis setup for Smart CÍdsS[ Ï^|].eccceee«eeecerkekikeerrsetriiiiiiririrrier 11 Figure 2. 4 Smart class with level Personal Devices[14 ] css 12 Figure 2. 5 Smart class with level Virtual Reality[ 14]. 7 Model smart class of Alo360 (tt)[ 15].
8 NVIDIA Jetson NANO[2 J crscsssssssssssssesesessnsaiesssesessssnsuesssesesessnsnseesseesessnnnnneeseete 19 Figure 2. 9 Graphic about how TensorRT WOrk[2]. 11 Three step of model mICiNN[Õ]. 12 Fully Connected Layer | Š |.eeec-ccccvvveeeeteeecerkekvvveeesetrerkkkkteraeisrrter 24 Figure 2.
13 Triplet-loss and learning[5]. 17 Disadvantages of WeblÑTC[ ]Ó ],. 1 Overview ArchiteCtureseesOO) Figure 3. 3 Facial jack recognition ALgOrithm vssOD Figure 3.
4 Algorithm diagram for facial recognition. S TOT algorithm đỉA9TGHH. 6 điagram [OT SIY€QIHÌH.eeeeec-cccccccceeeeetrererteeteeeerrirrrrrrrirrirrrssoooe OO Figure 4. 1 Successful identifying image (name Will be the SAME).
2 Images when the recognition fails (name will be different). 3 Images in good resolution, good condition and near distance. 4 Images in good resolution, good condition and far distance. 5 Images in good resolution, bad condition and near distance swe+5 Figure 4.
6 Images show identification after 200 times Figure 4. 7 Lights off when transmitting signal 0 from leVÌC€. 8 The light on when transmitting signal 1 from the deviCe. 9 Streaming lecture with 8 devices participating at the same time.
10 Streaming lecture with 12 devices participating at the same time. 11 Streaming video achieves HD with multiple conneCtiOns. 12 If there are no students in the class, the are no image. 13 If there are students in the class, the image Will appeqF.
14 Image of teacher subjects in the feature CXAMsissies DS Figure 4. 15 Teacher can create an exam during the lecture through feature exam a 4 Figure 4. 16 Teacher can edit questions after they've created them crus5 TABLE CATEGORY Table 4. 1 Data table of dataset Ï,.
2 Data table of dataset 2 srsssssscsseresescsssesesssesssssesssesssssesssssssssssssssssssssssssssssesees OO Table 4. 3 Facial recognition result of use dataset] for test case 1. 4 Recognition speed with 1000 times .eee-c-ecc5ccSe+ -Ö42 Table 4. 5 Facial recognition result of use dataset2 for test case 5 with distance.
6 Recognition speed with 200 times in good condifiOH. 7 Facial recognition result in bad CONItION wissen40 Table 4. 8 Recognition speed with 200 times in bad CONGItION. 9 Data table results after 100 smart pulpit lights on and off via Wi-Fi.
10 Data table results streaming 8 devices participating at the same time 49 Table 4. 11 Data table results streaming 12 devices participating at the same time 50 Table 5. 1 Overview about the function of the smart PUIPit cssD7 Table 5. 2 Overview of the hardware used in the smart pHÏDÍI.
2Ö CATEGORIES ACRONYMS CNN Convolutional Neural Network mtCNN Multi-Task Cascaded Convolutional Neural Networks IOT Internet Of Things MQTT Message Queuing Telemetry Transport QoS Qualities of services QR Quick response VR Virtual reality AR Augmented reality WebRTC Web Real-Time Communication GPU Graphics Processing Unit CPU Central Processing Unit ABSTRACT With the start of the 4.0 industrial revolution, the loT platform is being used in all areas around us. In the field of education, there is one such device that has been integrated with this technology and that is the smart pulpit. Smart pulpits have been used for a long time in many famous universities around the world such as Oxford, MIT. with functions such as lectures presentations, devices control in the classroom as well as being able to do all of that on the device.
In Vietnam, there are a few products with similar features but not many improvements and the disadvantage are that the price is very high and cumbersome to use, it does not bring high flexibility and has not been able to apply many technologies in it. According to our findings, there is no research published on applying artificial intelligence to the pulpit to be able to easily identify teachers in class as well as adding an improved function of combating academic fraud. The application of artificial intelligence can help prevent cheating in exams as well as learning. At the same time, the implementation of the system on a compact device can help teachers connect with students wherever they are expressing the flexibility of the product and adapting to many different learning conditions, especially in the situation of online learning is growing today.
Based on the results of the survey, our team proposed to carry out the project study " Smart Pulpits applying IOT platform to smart classrooms ". The results of the project will contribute to the creation of a domestic product with many improved functions such as teacher facial recognition as well as validate students to prevent fraud. The project can be applied to all schools in the future. INTRODUCTION - Reasons for implementing the topic To help teachers apply technology to teaching with simple and intuitive operations, applying 4.0 technology as well as AI.
Our team decided to research and develop a smart pulpit device with functions such as facial recognition, using IoT to be able to connect devices in the classroom and especially to be able to stream lectures remotely in the current epidemic situation. - Goal Create a physical product in a smart classroom to be able to present a few functions such as facial recognition, device control in the classroom, automatic attendance, streaming,. To accomplish the set goal, the following tasks should be performed: e Build a facial recognition model. e Build an IoT system with inter-linked nodes.
e Build an image streaming system using WebRTC. e Create a product interface. e Integrating the above functions into the product. e Building experimental environments and experimental products.
- Subjects of study e Method of identifying the face in the photo. e Facial recognition methods. e MQTT protocol for IOT device control. e WebRTC connection for lecture presentations.
e QT platform for product interface building. - Scope of research Content 1: Document research e Study relevant documents and develop a plan to implement the topic. e Study image processing algorithms related to the scope of topics. e Evaluate the results of research works and the applicability to the problem of the topic.
Expected results: Overview report and results of evaluation of related topics. Content 2: Build a facial detection program in an image e Pre-processing of photos before identification processing. e Partition objects on photos. e Recognize face area.
Expected result: zoning the face in the image. Content 3: Build a facial recognition program e Extract face embedding. e Compare data with know faces in database. Expected result: Facial recognition.
Content 4: Building a system that connects devices in class e Create sever MQTT so that nodes can be public and subscribe. e Client sends signals through broker to be able to transmit signals to other clients. e Client receives the signal sent and conducts control of the device through the received signal. Expected result: Remote control of devices via Wi-Fi.
Content 4: Build a slideshow system using WebRTC e Create a broadcast sever. e Presentation pulpit through sever created. e Clients can connect to sever to access the streaming system through the tables. Expected results: lectures are shown and students can watch live - Topic limitations e Achieve at least 90% Recognize success rate.
e Voice recognition not yet been implemented. e Experimental lice with 2 devices and classroom items on a small scale. e Not able to stream sound yet. e Streaming with max connection is 15.
e Build a demo program with a simple interface. International research Smart classrooms have become increasingly popular in the world with the great development of technology as well as being aware of the importance of education today. With the possible future trend, there has been a lot of research on smart classrooms in the world, so we compare the pros and cons of this type of class. Some typical studies: - Smart Classes and Their Importance in Imparting Education of written by ANU_V in Embibe.com [12] What is Smart Class? Smart classrooms are modern methods that provide quality education to the students for better learning outcomes with the help of technology.
Traditional classrooms are losing their effectiveness as all schools are adapting smart classrooms in their educational institutions. Smart classrooms are equipped with the latest technology such as visualizers, smartboards, laptops/computers, Internet connectivity, projectors, amplifiers, speakers, pulpits, microphones and many more. As you know, nowadays students are inclined towards gadgets and technology such as mobiles, tablets, computers, etc. They prefer to use technological tools for sharing and transferring study materials for their learning.
With this, learning is becoming more fun and interesting for the students along with complete attention in the classroom. Smart classrooms are rapidly changing the way teachers and students are experiencing learning activities with innovative technology. Technology is very important today in the world and students have plenty of tools to gain knowledge from. Based on the above ANU_V divided into 3 types of Smart classrooms e Basic smart classrooms: equipped with gadgets like computers, laptops, projectors, viewing screen, etc.