MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION GRADUATION PROJECT ELECTRICAL AND ELECTRONIC ENGINEERING TECHNOLOGY SMART HOME ASSISTANT SYSTEM LECTURER: PhD. VU VAN PHONG STUDENT: TRINH MINH DUC PHAM MINH NHAT SKL 009337 Ho Chi Minh City, August 2022 HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING GRADUATION PROJECT SMART HOME ASSISTANT SYSTEM Done by: TRINH MINH DUC - 17151011 PHAM MINH NHAT - 17151022 Major: ELECTRICAL AND ELECTRONIC ENGINEERING TECHNOLOGY Advisor: VU VAN PHONG, PhD. Ho Chi Minh City, August 2022 HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING GRADUATION PROJECT SMART HOME ASSISTANT SYSTEM Done by: TRINH MINH DUC - 17151011 PHAM MINH NHAT - 17151022 Major: ELECTRICAL AND ELECTRONIC ENGINEERING TECHNOLOGY Advisor: VU VAN PHONG, PhD. Ho Chi Minh City, August 2022 THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- Ho Chi Minh City, August 15, 2022 GRADUATION PROJECT ASSIGNMENT Student name: _________________________ Student ID: ___________________ Student name: __________________________ Student ID: ___________________ Student name: __________________________ Student ID: ___________________ Major: ________________________________ Class: ________________________ Advisor: ____________________________ Phone number: _________________ Date of assignment: _____________________ Date of submission: _____________ 1.
Initial materials provided by the advisor: ___________________________________ 3. Content of the project: _________________________________________________ 4. Final product: ________________________________________________________ CHAIR OF THE PROGRAM ADVISOR (Sign with full name) (Sign with full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- Ho Chi Minh City, August 15, 2022 ADVISOR’S EVALUATION SHEET Student name:. Content of the project:.
Approval for oral defense? (Approved or denied) .) Ho Chi Minh City, August 15, 2022 ADVISOR (Sign with full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- Ho Chi Minh City, August 15, 2022 PRE-DEFENSE EVALUATION SHEET Student name:. Name of Reviewer:. Content and workload of the project. Approval for oral defense? (Approved or denied) .) Ho Chi Minh City, August 15, 2022 REVIEWER (Sign with full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- EVALUATION SHEET OF DEFENSE COMMITTEE MEMBER Student name:.
Name of Defense Committee Member:. Content and workload of the project .) Ho Chi Minh City, month day, year COMMITTEE MEMBER (Sign with full name) Acknowledgements First of all, we would like to offer our deepest gratitude and respect toward Ph. Vu Van Phong, the one who give us many valuable viewpoints in the time we carry out our project. We truly thank you for working together and guiding us throughout the time of researching and completing this project.
Although there are many hardships and errors, we still manage to overcome them thanks to Ph. Vu Van Phong vast experience and accurate instructions so that our team can improve and complete the project in time. We also would like to express our sincere appreciation to the Board of Directors of Ho Chi Minh City University of Technology and Education and the teachers in the faculty of Electrical - Electronic Engineering and Faculty for High Quality Training, those who have taught and created all favorable conditions and environment to help us during the process of learning and executing our project. Student Table of contents Chapter 1: INTRODUCTION.
2 Chapter 2: LITERATURE REVIEW.1 Smart Home System Overview .3 Voice Recognition Relevant Theory.2 Voice Recognition Method .4 Face Recognition Relevant Theory.5 Face Recognition Method .2 Human identification at a distance (HID) .1 Raspberry Pi OS. 12 Chapter 3: HARDWARE DESIGN .2 Logitech Webcam C270 HD .2 Main AI System .3 Face Recognition System .3 Create data set .4 Locate faces and encoding .5 Locate faces and compare in real time .5 Python HTTP Server. 38 Chapter 5: RESULT AND EXPERIENCE. 42 Chapter 6: CONCLUSION AND RECOMMENDATION.
52 Main AI code:. 52 Face Recognition code:. 96 List of Figures Figure 1.1: Smart home illustration .1: Smart home system .6: Raspberry Pi OS .2: Raspberry Pi 4 diagram .3: Logitech Webcam C270 HD .7: Bluetooth Speaker Fenda W5 Plus .9: System wiring diagram .1: AI system flowchart .4: Registered users data.5: Code to read the data.6: Code to encoding data .7: Code to locate faces and compare in real time .9: Real-time face capture .10: Code for create the green box .11: Stranger image files .12: Data log files .14: GerPan application interface .15: HTTP server communication .17: Communicating through web browser .18: HTTP server receiving request .19: Code for controlling devices .1: The front of the model .2: Smart home model .3: Sideway of the model.5: Time result in English .6: Time result in Vietnamese .7: Date result in English .8: Date result in Vietnamese .9: Weather result in English .10: Weather result in Vietnamese .11: Schedule result in English .12: Schedule result in Vietnamese .13: Light on/off result in English .14: Light on/off result in Vietnamese .15: Lights and fans are on .17: The user in the system.19: Face recognition log. 49 List of Tables Table 3.1: Raspberry Pi 4 Specification .2: Logitech Webcam C270 HD Specification .3: 4 Relay Module Specification .5: DC Motor Specification .6: Bluetooth Speaker Fenda W5 Plus Specification.7: Servo SG90 Specification .1: Voice commands result .2: Face recognition testing result.
49 Abstract The project aims to create a smart home assistant system with convenient features for the user. With a basic Artificial Intelligence (AI) written with python on a Raspberry Pi 4, the system can understand commands from the user voice. From asking simple questions about date, time and weather, the user can also turn on/off devices in the house. An app is also developed to control the devices as well.
A face recognition system is also used to detect the owners and to send messages to the app mentioned above. Keywords: AI, smart home, voice recognition, face recognition, text-to-speech, speech-to-text Chapter 1: INTRODUCTION 1.1 Problem With the advancement of modern technology, people also gradually seek more convenient things for their lives. Since then, all kinds of automatic technology began to form and develop non-stop. These technologies bring comfort and convenience to the user so it is very popular.1: Smart home illustration Therefore, our team decide to study the topic “Smart home assistant system”.
Using a Raspberry Pi 4 as the main CPU, our team will build a model representing a smart home with functioning features.2 Objective The purpose of this topic is to build a model that can represent a real smart home with user’s voice as input to command the assistant to ask/answer questions and control the devices. We also build an application to also control the devices in the house and a face recognition system to detect the owner.3 Constraints This prototype is designed for representing the real product only. This prototype only uses the low cost, easy to use materials and device available on the markets. We will use small LEDs and motors as devices in the house.
This system will only working with predefined features with given keywords. The system is researched and studied for reality experience, which is not after or applied to any specific system or model.4 Research Method Conduct research about basic AI, voice recognition, face recognition. Construct, simulate and analyze the system operation.5 Outline Chapter 2: LITERATURE REVIEW Analyzing smart home systems and techniques available nowadays. Mentioning some basic information and knowledge about relevant techniques, software and concepts used in this project.
Chapter 3: HARDWARE DESIGN The model design with parameters and wiring diagram are presented. Selecting devices that are suitable and available on the market for this project. Chapter 4: METHODOLOGY Illustration of the system operation in the form of flow chart and how the system looks like when running. Chapter 5: RESULT AND EXPERIENCE Presenting result of the built prototype, result of operation testing and giving analysis.
Chapter 6: CONCLUSION AND RECOMMENDATION Discussing advantages and disadvantages of the system and giving some recommendations for further improvement. 2 Chapter 2: LITERATURE REVIEW This chapter will illustrate some of the existing research model and method about basic AI, voice recognition, face recognition. In addition, there are also some comparisons to be discussed in order to highlight the advantage of this topic.1 Smart Home System Overview A smart home means your home encompasses a savvy home system that interfaces together with your apparatuses to computerize particular errands and is ordinarily remotely controlled. You'll utilize a shrewd domestic framework to program your sprinklers, set and screen your domestic security framework and cameras, or control machines like your fridge or discuss conditioning and warming.1: Smart home system Smart home devices are not continuously vitality productive, but most can be utilized in ways that assist your moderate vitality.
After you have shrewd domestic apparatuses, lights, or a shrewd indoor regulator, you control when and how these will turn on and work. Subsequently, you've got more control over your vitality utilization and are more likely to alter your vitality utilization propensities.2 AI Introduction According to the father of Artificial Intelligence, John McCarthy, it is “The science and engineering of making intelligent machines, especially intelligent computer programs”. Artificial Intelligence is a way of making a computer, a computer-controlled robot, or a computer program think intelligently, within the comparative way the shrewdly people think.2: Artificial intelligence AI is accomplished by examining how human brain considers, and how people learn, choose, and work whereas attempting to solve an issue, and after that utilizing the results of this ponder as a premise of creating cleverly program and frameworks. Artificial intelligence is a science and innovation based on disciplines such as Computer Science, Science, Brain research, Etymology, Science, and Engineering.
A major pushed of AI is within the improvement of computer capacities related with human insights, such as thinking, learning, and issue tackling.3: AI AI has been dominant in various fields such as: Gaming: AI plays crucial role in strategic games such as chess, poker, tic-tac-toe, etc., where machine can think of large number of possible positions based on heuristic knowledge. Natural Language Processing: It is possible to interact with the computer that understands natural language spoken by humans. Expert Systems: There are some applications which integrate machine, software, and special information to impart reasoning and advising. They provide explanation and advice to the users.
Vision Systems: These systems understand, interpret, and comprehend visual input on the computer. For example: - A spying aero plane takes photographs, which are used to figure out spatial information or map of the areas. - Doctors use clinical expert system to diagnose the patient. - Police use computer software that can recognize the face of criminal with the stored portrait made by forensic artist.
Speech Recognition: Some intelligent systems are capable of hearing and comprehending the language in terms of sentences and their meanings while a human talks to 5 it. It can handle different accents, slang words, noise in the background, change in human’s noise due to cold, etc. Handwriting Recognition: The handwriting recognition software reads the text written on paper by a pen or on screen by a stylus. It can recognize the shapes of the letters and convert it into editable text.
Intelligent Robots: Robots are able to perform the tasks given by a human. They have sensors to detect physical data from the real world such as light, heat, temperature, movement, sound, bump, and pressure. They have efficient processors, multiple sensors and huge memory, to exhibit intelligence. In addition, they are capable of learning from their mistakes and they can adapt to the new environment.3 Voice Recognition Relevant Theory 2.1 Introduction Voice or speaker recognition is the capacity of a machine or program to get and decipher transcription or to get it and carry out talked commands.
Voice recognition frameworks enable consumers to connect with innovation basically by talking to it, empowering hands-free demands, updates and other straightforward assignments.2 Voice Recognition Method Voice recognition software on computers requires that analog sound be changed over into advanced signals, known as analog-to-digital transformation.