VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY ADVANCED PROGRAM IN INFORMATION SYSTEMS THAI HOANG THINH LE TRINH QUANG TRIEU HIGH PERFORMANCE SEARCH AND VERIFICATION SYSTEM FOR COVID-19 INFORMATION BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS HO CHI MINH CITY, 2021 NATIONAL UNIVERSITY HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY ADVANCED PROGRAM IN INFORMATION SYSTEMS THAI HOANG THINH -17521091 LE TRINH QUANG TRIEU - 1751314 HIGH PERFORMANCE SEARCH AND VERIFICATION SYSTEM FOR COVID-19 INFORMATION BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS THESIS ADVISOR PHD. LE KIM HUNG HO CHI MINH CITY, 2021 ASSESSMENT COMMITTEE The Assessment Committee is established under the Decision. by Rector of the University of Information Technology. ACKNOWLEDGMENTS ĐẠI HỌC QUỐC GIA TP.
HÒ CHÍ CỘNG HÒA XÃ HỘI CHỦ NGHĨA MINH VIỆT NAM TRƯỜNG ĐẠI HỌC CÔNG Độc Lập - Tự Do - Hạnh Phúc NGHỆ THÔNG TIN TD HCM, ngày 26 tháng 12 năm 2021 COMMENTARY OF THE GRADUATION (BY MENTOR) Thesis title: HIGH PERFORMANCE SEARCH AND VERIFICATION SYSTEM FOR COVID-19 INFORMATION Students work: Mentor Thai Hoang Thinh 17521091 TS. Lé Kim Hung Lê Trinh Quang Triệu 17521314 Thesis review 1. About the report: Number page: Number chapter: Number of data tables: Number of figures: Number of references: Product: Some comments on the report format: 2. About research content: 3.
About the application program: 4. About the student's working attitude: General assessment: Thesis pass/fail the requirements of an engineering/bachelor's thesis, graded Excellent/Good/Average. Score each student: Thái Hoang Thinh. 10 Lê Trịnh Quang Triệu.
Lé Kim Hung ĐẠI HỌC QUOC GIA TP.HÒ CHÍ — CỘNG HÒA XÃ HỘI CHỦ NGHĨA MINH VIỆT NAM TRƯỜNG ĐẠI HỌC CÔNG Độc Lập - Tự Do - Hạnh Phúc NGHỆ THÔNG TIN TD HCM, ngày 26 tháng 12 năm 2021 COMMENTARY OF THE GRADUATION (BY REVIEWER) Thesis title: HIGH PERFORMANCE SEARCH AND VERIFICATION SYSTEM FOR COVID-19 INFORMATION Students work: Reviewer Thai Hoang Thinh 17521091 Lê Trinh Quang Triệu 17521314 Thesis review 2. About the report: Number page: Number chapter: Number of data tables: Number of figures: Number of references: Product: Some comments on the report format: 3. About research content: 4. About the application program: 5.
About the student's working attitude: General assessment: Thesis pass/fail the requirements of an engineering/bachelor's thesis, graded Excellent/Good/Average. Score each student: Thái Hoang Thinh. /10 Lê Trịnh Quang Triệu. 190 Reviewer Thank you First of all, we would like to thank all the instructors of the Department of Information System, as well as the instructors at the University of Information Technology - Vietnam National University, Ho Chi Minh City, who have provided us with invaluable knowledge, lessons and experience over the past four years.
We also spent a little time and arranged the schedule to have the best opportunity to complete the graduation thesis. Wishing the Faculty of Information Systems in particular and the University of Information Technology in general to continue to achieve success in teaching and training talents, is a firm belief for generations. students on their educational journey. We would like to express our heartfelt gratitude to Dr.
Le Kim Hung, our lecturer, in particular. He has always cared about and helped us solve problems and difficulties in the implementation process, thanks to the valuable experiences and lessons he has shared. We were able to successfully complete this graduation thesis thanks to you. Next, we'd like to express our gratitude to our family for their unwavering support and encouragement throughout our time at the University of Information Technology - Vietnam National University in Ho Chi Minh City, which has given us more energy to attend classes.
Finally, our team would like to thank the brothers, sisters and students at the University of Information Technology - Vietnam National University, Ho Chi Minh City, who have always enthusiastically supported, shared ideas and suggestions for us in the future. Ho Chi Minh, date 26 month 12 2021 Thai Hoang Thinh - Lé Trinh Quang Triéu ĐẠI HOC QUOC GIA TP. HO CHÍ CONG HOA XA HOI CHU MINH NGHIA VIET NAM TRUONG DAI HOC CONG NGHE Độc Lập - Tự Do - Hạnh Phúc THÔNG TIN DETAILED OUTLINE THEME NAME: HIGH PERFORMANCE SEARCH AND VERIFICATION SYSTEM FOR COVID-19 INFORMATION Mentor: TS. Lé Kim Hing Time to do: From 07/08/2021 to 26/12/2021 Students: Thai Hoang Thinh - 17521091 Lé Trinh Quang Triéu - 17521314 Content of the subject: Objectives of the study: Apply FAISS and Fake News Detection Model available.
Crawling data from rss feed. Learn about gRPC's HTTP/2 protocol and its application to the system. Use Elasticsearch to quickly query data. Testing, building and optimizing the website system to provide information about Covid-19 has been confirmed.
Management services by Protainer. + Deploy system and public. 10 performance search and verification system for covid-19 information. Research scope: Research on FAISS and Fake News Detection model, gRPC protocol and find suitable datasets.
Methods of implementation: e Theoretical basis: + Crawl data on reputable rss feed and store data on Elasticsearch. Integrated FAISS finds the closest semantic data index and Fake News Detection model to calculate the correct percentage of the sentences searched. + Use gRPC protocol to request data quickly. Expected results: There is a large and validated data set.
You may rate the search sentence as low comprehension. Create a system to check whether the information entered is correct or not. Create a scalable, stress-resistant and horizontally scalable system (increasing the number of nodes). Technology related to the topic: Use Elasticsearch to bookmark index data.
Use docker compose and docker swarm. Available models such as Faiss and Fake News Detection. Crawl data using selenium with rss feed. Deployment using docker and load balancing using nginx.
11 e Implementation plan: 3/8/2021 - 24/8/2021 Learn about python and tensorflow. 25/8/2021 - 25/9/2021 Crawl and find the dataset. 26/9/2021 - 30/10/2021 Research FAISS Model. Research Fake News Detection Model.
Research Roberta base Model. 30/10/2021 - 31/10/2021 Research about servers. 12 Set up a private server to train models. 31/10/2021 - 2/11/2021 Research domain IP.
Use domain to map to server. - Hide IP with cloudflare. 16/11/2021 - 26/12/2021 - Deploy all systems to server and public outside by docker swarm. - Check system operation Confirmation of mentor TP.
HCM, 26/12/2021 Student 1 Student 2 Ts. Lê Kim Hùng 13 TABLE OF CONTENTS TABLE OF CONTENTS 15 LIST OF FIGURES 16 Chapter 1 PROBLEM STATEMENT 1 Introduction 1 Rationale 1 Aims and Objectives 2 Chapter 2 LITERATURE REVIEW AND THEORETICAL BACKGROUND 4 Theoretical background 4 Crawler api 4 Dataset 5 Fake news detection 6 FAISS 8 Semantic similarity in universal sentence encoder. 8 Elasticsearch engine 10 How does elasticsearch work 10 Reindex in elasticsearch. 10 gRPC protocol 12 HTTP/2 employs binary rather than text 13 Multiplexing of Requests and Responses.
13 Streams 13 How to connect a gRPC server. 15 CI/CD github 16 Docker engine 17 Docker container 17 Docker swarm 18 Docker compose 20 Portainer management 21 Domain ip address 22 15 Chapter 3 MODEL ARCHITECTURE 24 General 24 Overview 24 Workflow 26 Detailed training process 26 Detailed predicting process 26 Crawler API 27 Workflow 27 Fake News detection API 30 User Interface 32 Deployment and load balancing in docker swarm 33 Chapter 4 IMPLEMENTATION 35 RSS-crawler API 35 Fake News detection api 44 User interface 52 Github CI/CD process 60 Elasticsearch 64 Deploy process 69 Chapter 5 CONCLUSION 72 REFERENCES 73 Articles 73 Websites 73 16 LIST OF FIGURES Figure 2-1 Workflow RSS work. +fDoOnF Figure 2-2 Flow RSS crawler API work. Figure 2-3 Dataset sample 1 Figure 2-4 Dataset sample 2 Figure 2-5 Predict Process Workflow Figure 2-6 Summary of all models and performances Figure 2-7 Example Encoder data by encoder.
10 Figure 2-8 Example Encoder data by encoder. 10 Figure 2-9 Use FAISS search something. in Figure 2-10 Flow Elasticsearch engine work. 12 Figure 2-11 Two Sentence Input Elasticsearch 12 Figure 2-12 Inverted Index Elasticsearch 13 Figure 2-13 Search Query Elasticsearch 13 Figure 2-14 gRPC workflow 14 Figure 2-15 gRPC auto convert text, image, etc to binary 14 Figure 2-16 Stream gRPC workflow 15 Figure 2-17 Example Compare HTTP/1.1 and HTTP/2 16 Figure 2-18 Core Executed Envoy 16 Figure 2-19 Envoy Translates HTTP/* To HTTP/2 16 Figure 2-20 Flow CI/CD work in github (Viblo CI/CD) 17 Figure 2-21 Flow docker work (Topdev docker) 18 Figure 2-22 Swarm node (Bizflycloud) 20 Figure 2-23 Workflow docker compose build (Azuremarketplace) 21 Figure 2-24 Dashboard of portainer example 22 Figure 2-25 List activity of container example 22 17 Figure 2-26 Register domain complete at matbao 23 Figure 2-26 Grant DNS domain 24 Figure 3-1 Overview model architecture 25 Figure 3-2 Overview public system 26 Figure 3-3 Crawler process 28 Figure 3-4 Al-core overview 29 Figure 3-6 Fake News detection API workflow 32 Figure 3-6 Encoding Copus Workflow 33 Figure 3-7 Result classification sentences 34 Figure 3-8 Envoy transfer HTTP/1* to HTTP/2 and general protoc.
35 Figure 3-9 Apply nginx in docker swarm for management. 36 Figure 4-1 Swagger UI 38 Figure 4-2 Test uncrawlable url 39 Figure 4-3 Test crawlable url 40 Figure 4-4 Test add url to RSS-crawler api 41 Figure 4-5 Test remove rss url 42 Figure 4-6 Test get all rss-url in data-base 43 Figure 4-7 Swagger UI 44 Figure 4-8 Demo train a bunch of data include a random sentence 45 Figure 4-9 Result after call predict api 46 Figure 4-10 Saving checkpoint api 41 Figure 4-11 Where to save checkpoint 48 Figure 4-12 Code structure 49 Figure 4-13 Libraries in package Json 50 Figure 4-14 Atomic design workflow 51 Figure 4-15 Source code 52 Figure 4-16 Train.proto 53 18 Figure 4-17 serve.proto 55 Figure 4-18 All file js have been general.yaml 57 Figure 4-20 Test gRPC on BloomRPC. 58 Figure 4-21 Function call gRPC on ReactJS 58 Figure 4-22 Website CoronaCheck Information 59 Figure 4-23 CI/CD build and push basic flow 59 Figure 4-24 CICD steps 60 Figure 4-25 CICD code example 61 Figure 4-26 CICD process summary 62 Figure 4-27 Workflow history 63 Figure 4-28 Model overview 64 Figure 4-29 Status checker 65 Figure 4-30 Node checker 65 Figure 4-31 Kibana UI node checker 66 Figure 4-32 Example Network configuration 66 Figure 4-33 Example for kibana container 67 Figure 4-34 Example for elasticsearch master node 67 Figure 4-35 Initialize swarm 68 Figure 4-36 Swarm node checker 68 Figure 4-37 Swarm service result 69 Figure 4-38 Result after start up 69 Figure 4-39 Demo run a docker container in existed swarm 70 Figure 4-40 The swagger result after start up append 71 19 ABSTRACT Everyone in the world has been concerned about the covid-19 pandemic since the outbreak. At that time, it must be stated that in a pandemic, the media is the most important; if one news media is not reliable, a country can suffer severe economic consequences.
Since then, many unreliable sources of information have been widely disseminated on the internet, particularly on widely accessible social media platforms. Fake news is distributed by social media and news organizations to increase readership or as psychological warfare. The goal, according to Inge Tang, is to profit from clickbaits. With flashy titles or designs, clickbaits entice users to click links in order to boost ad revenue.
Because of advances in communication brought about by the rise of social networking sites, this exposure is due to the prevalence of fake news. The project's goal is to develop a solution that users can use to identify and filter out websites that contain false or misleading information. In this graduation thesis, I investigate and comprehend the problem of detecting fake news. In addition, I used my existing knowledge, as well as learning more about machine learning and deep learning, to use available models and build a website system to detect fake news covid-19.
20 Chapter 1 PROBLEM STATEMENT 1. Introduction In 2019, a virus was discovered in Wuhan, China, but it could not be contained and spread throughout the world.