VIET NAM NATIONAL UNIVERSITY HCMC UNIVERSITY OF INFORMATION TECHNOLOGY ADVANCED PROGRAM IN INFORMATION SYSTEMS AN ARTIFICIAL INTELLIGENCE-BASED CHATBOT FOR UIT FACEBOOK PAGE BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS HO CHI MINH CITY, 2020 VIET NAM NATIONAL UNIVERSITY HCMC UNIVERSITY OF INFORMATION TECHNOLOGY ADVANCED PROGRAM IN INFORMATION SYSTEMS TRAN VAN HOÀNG - 16520449 VÕ HONG NHAT - 16520887 BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS THESIS ADVISOR PH. Nguyén Thanh Binh HO CHI MINH CITY, 2020 ASSESSMENT COMMITTEE The Assessment Committee is established under the Decision. by Rector of the University of Information Technology. ACKNOWLEDGEMENTS First and foremost, we would like to express our grateful attitude to the entire Information System faculty staff for helping us since we set our foot in this school.
During the time since my first year up to now, we have received a lot of help, not only from teachers but also from the infrastructure staff. In particular, we would like to thank Dr. Nguyén Thanh Binh for his help throughout the realisation of this thesis, especially concerning its redaction as well as for initially presenting this thesis’ subject during one of his academic lessons. We would also like to personally thank the Chatbot Big Data team at FPT Telecom for being an incredible support during the whole development of this thesis as well as for providing the adequate tools necessary for the achievement of this work.
In particular, I would like to thank Mr. Trần Xuân Hậu - my leader and Mr. Dang Minh Chương for their help. Finally, a big thank you goes to my friends and family who were an incredible support for me during the redaction of this thesis.
Once again, we sincerely thank you. TABLE OF CONTENTS cae ABSTTRACCTT.- -s - s 9 TnHHh nHnngh nh 4 1.- -- G11 TH ng ng 5 1. Popular types of ChafOf.- ác + 231g HH ng ni, 5 1. Natural Language Processing in chatDOt.
Chatbot building DFOC€SS.- --- + 31183111311 9 11191111 vn rệt 7 OBJECTIVES AND GOALU. có 5c cọ HH CC 001 0080050896 11 2. Evaluate and find Solutions. Ta cổ À—⁄É_ LÔ À À («St hsktseksseesetrseessekssee 11 TECHNICAL BACKGROUND.
Gv HH HH TH HH hệt 12 1. Rasa ArChIt€CfUTG. LG TH TH TH TH HH HH 14 1. Natural Language Understanding.
HH HH HT HH như, 15 3.AI Conversation PÏatÍOTI.-- -- <5 + E311 E ke 16 THEORETICAL BACKGROUND. Go HH HH 030 19 4. Word cac on e. Text modelling and features.
Intent clasSIÍICafIOT. Named Entity (voi. SYSTEM DESIGN AND ANALYSTIS. HH HH TH HH HH HH gi.-- Ăn HH HH ng 25 5.
TH HH ng ng Hệ 25 0 Table “tkb?. Q0 HT ng HT HH nhện 26 5. FAQ feature oo. Report feature nh .c G0 13065083 50 39 BI; 9‹io0i6)9) 220i i(cc:aadđa.
Rasa NLU mO€]Ï.- 5 <6 + E1 E1 991 93 E1 1v nh ng ng ệt 43 6. Parsing timetable InÍOrImatfIOI. SH TH TH ng kg 47 6. By COUTS€ NAIT.
Ăn SH TH TH HH Hi,49 6. Installation and deployMent. EVALUATION AND CONCLUSION.- s5 G101 90 ng Hi như52 7. Possible improvements and future WOTKS.- - - s11 TH TH HH kg rry 55 REFERENCE SG .cscscssssssscsscsssscsrsesscssesssescseseseccsssessessssessesseseseasseseseasseseseessesesees 56 LIST OF FIGURES caso Figure 1.1: UIT’s students needs for chatbot 2 Figure 1.2: UIT’s students needs for chatbot’s functionality 3 Figure 1.3: UIT’s students needs for chatbot’s platform 3 Figure 1.4: Popular types of chatbot 6 Figure 4.1: High-level of Rasa architecture 16 Figure 4.AI Conversation Platform working mechanism 20 Figure 5.1: Architecture of chatbot application 22 Figure 5.2: Data flow diagram 23 Figure 5.3: Business Function Diagram 26 Figure 5.4: Welcome carousel 27 Figure 5.5: Overview of timetable feature 29 Figure 5.6: Conversation flow of timetable feature 29 Figure 5.7: Timetable feature activity diagram 30 Figure 5.
Checking user’s history chatlogs for Student ID 31 Figure 5. Returning timetable information based on provided Student ID32 Figure 5.10: Report feature 33 Figure 5.11: Overview of all Departments in report function 33 Figure 5.12: “Phong dao tao” and “Phong thiét bi” Department in the context of the chatbot application 35 Figure 5.13: Activity diagram of report feature 36 Figure 5.14: Sequence diagram of report function 36 Figure 6.1: Json API Card in FPT.AI Conversation Platform 37 Figure 6.2: Example of training format for intent ask_schedule 40 Figure 6.3: Training data intent distribution 41 Figure 6.4: Timetable data as JSON format 41 Figure 6.5: Rasa Component Lifecycle 42 Figure 6.6: Full Pipeline of Rasa NLU model 45 Figure 6.7: Training result over 300 epochs 45 Figure 6.8: Dimension supported by duckling 47 Figure 6.9: Example request of Broadcasting API 49 Figure 7.1: High-level overview of DIET model 52 Figure 7.2: In-depth overview of the DIET architecture 52 Figure 7.3: DIET architecture Pipeline 53 LIST OF TABLES ca Leo Table 5.1: Table mssv description 24 Table 5.2: Table tkb description 24 Table 5.3: Table report description 25 Table 5.4: All Departments along side with their website at UIT 34 Table 6.1: Backend routes implemented in the chatbot application 38 Table 6.2: Rasa NLU Components used in Pipeline 43 Table 6.3: Parameters of Broadcasting API request 49 LIST OF ABBREVIATIONS 3 Leo UIT University of Information Technology NLP Natural Language Processing NLU Natural Language Understanding DIET Dual Intent Entity Transformers BoW Bag-of-words SVM Support Vector Machine ABSTRACT Nowadays, Chatbot is a trend for automating communication between user and server. Whoever is at the forefront of using Chatbot will have more opportunities in their hands regardless of the profession, the University of Information Technology (UIT) is no exception. Since UIT has so many types of information that come from many portals, it is difficult for students, parents or even lecturers to find information when facing problems.
Information is scattered all over the forums, websites and social networks. When they have problems while studying or working at UIT, they will be confused since they do not know where to find information or ask for help from which Department. Another issue is that every time students need to get timetable information, they have to go to the Office of Academic Affairs website (https://daa.vn/), login using their student account (and they must solve the CAPTCHA as well). This is a complex and time-consuming process.
By being aware of such issues, we decided to design and implement a Chatbot system and integrate it on the most popular social network today — Facebook, to be a communication tool between the school and students as well as others. UIT's chatbot will provide the most basic and diverse information to users such as timetable and general information about schools and faculties. The Chatbot can also provide timetable information with just one click. In conclusion, we think this project will help students, parents and lecturers a lot, not only by helping with finding information more easily and getting contact with suitable departments when facing problems.
INTRODUCTION This chapter will briefly introduce the context in which this work takes place in section 1. Then, the survey result we conducted will be given in section 1. A definition of chatbot in section 1. Applications of the field of conversational agents will be recalled in section 1.
Afterwards are some popular types of chatbot in section 1. Finally, some Natural Language Processing (NLP) keywords and specific knowledge will be given in section 1.6 and chatbot building process in section 1. Context After over 10 years of founding and development, University of Information Technology (UIT) includes 8 Faculties, 10 Administrative Offices, 7 Centers - Laboratories and 3 Unions. Every unit has their own website for storing information.
Traditionally, students or lecturers can post their questions or problems in the UIT forum to get answers, or they can come directly to suitable Departments for their problem. But sometimes, the information is too big and they don’t know exactly where to go, or where to post their problems, especially for parents and high school students who are looking for admission information. So that, we need a portal, or a channel capable of connecting all the information that stretches all over websites of UIT, especially on the online scene where most users are extremely demanding both in terms of response time and quality of the answers given. In order to provide students, parents and lecturers a portal to get information quickly and correctly.
In particular, we would like to deploy a dialogue system solution, also known as chatbot, that would be integrated seamlessly in the UIT Facebook page. The goal of this work is to design such systems. Survey Before designing and implementing the chatbot application, we conducted a survey in two weeks to collect student’s needs and feedback for the chatbot application. Bạn nghĩ Fanpage UIT nên hỗ trợ Chatbot không? 18 responses @co @ Không @ Sao cũng được Figure 1.1: UIT’s students needs for chatbot Ban muốn Chatbot co những chức năng nào? 16 responses Học phí Kết quả học tập Điểm rèn luyện Thời khóa biểu Lịch thi Cac biểu mẫu Qui chế nhả trường Figure 1.2: UIT’s students needs for chatbot’s functionality Ban mong muốn Chatbot hỗ trợ trên nền tảng nào? 15 responses Trang web nha trường 8 (53.3%) Fanpage Facebook 15 (100%) Zalo 5 (33.3: UIT’s students needs for chatbot’s platform 1.
Chatbot: A definition According to the Oxford English Dictionary, a chatbot is defined as follows: chatbot (n.): A computer program designed to simulate conversation with human users, especially over the Internet. In the scientific literature, chatbots are more formally referred to as conversational agents. In the context of this document, the terms chatbot/conversational agent will be used interchangeably. The underlying principle of every chatbot is to interact with a human user (in most cases) via text messages and behave as though it was capable of understanding the conversation and reply to the user appropriately.
The origin of computers conversing with humans is as old as the field of Computer Science itself. Indeed, Alan Turing defined a simple test referred to now as the Turing test back in 1950 where a human judge would have to predict if the entities they are communicating with via text is a computer program or not. However, this test’s ambition is much greater than the usual use case of chatbots; the main difference being that the domain knowledge of a chatbot is narrow whereas the Turing test assumes one can talk about any topic with the agent. This helps during the design of conversational agents as they are not required to have a (potentially) infinite domain knowledge and can, as such, focus on certain very specific topics such as for instance helping users book a table at a restaurant.
Furthermore, another general assumption chatbot designers bear in mind is that users typically have a goal they want to achieve by the end of the conversation when they initiate an interaction with a chatbot. This then influences the conversation’s flow and topics in order to achieve the chosen goal. This can be exploited by developers since certain patterns of behavior tend to arise as a result. Therefore, the definition of a chatbot adopted for this document is a computer program communicating by text in a humanly manner and who provides services to human users in order to accomplish a well-defined goal.
Chatbot applications Chatbot is created to support humans in the customer service field at the most basic level with repetitive, mundane tasks. Therefore, businesses will be able to reduce their human resources pressure, and their consultant teams can focus on solving more complicated and urgent tasks. Chatbot is applied in different ways for different types of businesses. Major purposes of it includes: e Consult and answer frequently asked questions from customers 24/7 e Support marketing campaigns (send information on promotions, discounts, new products.) e Suggest, search, and report prices for products and services based on customer demands e Book appointments, tables, rooms, air tickets.
e Receive declarations and information for opening cards, bank accounts.