chương 1 chuyển từ “Chapter 1. Introduction” thành “Introduction”, chương 5 chuyển từ “Chapter 5. Conclusion and future works” thành “Conclusion and future works” (trang 1 và trang 46). (Thông tin về số thứ tự của các trang trong biên bản này nếu không có chú thích thêm tương ứng với số trang trong quyển Luận văn đã chỉnh sửa.) Ngày tháng năm 2023 Giáo viên hướng dẫn Tác giả luận văn CHỦ TỊCH HỘI ĐỒNG GRADUATION THESIS ASSIGNMENT Name: Nguyen Thi Mung Phone: +84.vn; mungyp98@gmail.com Class: CH2021A Affiliation: Hanoi University of Science and Technology Nguyen Thi Mung - hereby warrants that the work and presentation in this thesis performed by myself under the supervision of PhD.
Nguyen Thi Thu Trang. All the results presented in this thesis are truthful and are not copied from any other works. All references in this thesis including images, tables, figures, and quotes are clearly and fully documented in the bibliography. I will take full responsibility for even one copy that violates school regulations.
Student Signature and Name Nguyen Thi Mung ACKNOWLEDGEMENT First of all, I would like to express my sincerest and deepest gratitude to Ph. Nguyen Thi Thu Trang. With both her enthusiasm and patience, she helped me orient the topic, gave suggestions, instructed me in detail, and created the best conditions for me to complete this thesis. She is like a warm mother but also strict, sometimes like a friend so that we can easily confide and share our difficulties.
Under her guidance, I feel that I have improved a lot. I would like to express my sincere thanks to the leadership and teachers at Hanoi University of Science and Technology in general and in the School of Information and Communication Technology in particular, for giving me the opportunity to study in a new environment. useful and memorable school in my student life. I also want to thank my brothers and sisters, friends, students in laboratory 914, and partners.
Thank you to everyone for their detailed guidance, enthusiastic help and encouragement during my time in the laboratory as well as during my thesis work. Along with that, I would like to thank my friends inside and outside the School of Information and Communication Technology for their interest, sharing and help in the past time. Finally, I would like to express my sincere thanks to my family. I thank my family for always loving, caring, and being a spiritual support, a great source of motivation for me to overcome my difficulties and challenges.
Tuan - my love, for being there to encourage me in the most stressful times. In the process of making my graduation thesis, even though I have tried my best, it is still inevitable that mistakes will be made. I look forward to receiving suggestions from teachers and friends so that I will not encounter these errors in the future. Once again, I sincerely thank you! ABSTRACT Finding information is becoming more and more challenging as the amount of knowledge on the Internet is getting bigger and bigger.
Conventional search engines only return lists of short paragraphs or related links, which makes it difficult for users, especially those who lack experience and search skills. Therefore, it is essential to build a Question Answering system capable of quickly giving an accurate answer to a question. Therefore, the author proposes the topic "Question Answering in Vietnamese" with the goal of building a question- answering system applicable to Vietnamese, especially considering the input factor through the human voice. Previous studies have solved the problem with many different approaches, in which, the approach of using similar questions helps to store and deploy the system easily.
Data is an important factor in helping ensure output for the system. The thesis proposes a process of building data based on similar questions including 02 main steps: collecting data through two systems named Written Collection System and Speech Collection System and applying this process in building data with the Digital Transformation domain with the initial question-answer pairs provided by the Ministry of Information and Communication. Based on the built data, the thesis also evaluates the question answering models in two approaches: classification and comparing the similarity between questions. The results show that the models have high accuracy from 82- 94%.
In which, the SVM model has the highest accuracy. At the same time, the model size is not too large and the prediction time is fast, which is suitable for deployment in practice. The evaluation results also show that, the Automatic Speech Recognition (ASR) module affects the quality of the model by 3. In the future, the thesis aims to expand the initial questions based on the available documents, and at the same time, partially automate and create tools to support the data quality controller to evaluate the data for the model.
TABLE OF CONTENT INTRODUCTION. Goal and scope .1 Text classification algorithms .2 BERT language model. BUILDING A VIETNAMESE QA DATASET .2 The process of building a Vietnamese QA dataset. VIETNAMESE QUESTION ANSWERING MODEL, EXPERIMENT AND EVALUATION.1 Vietnamese Question Answering problem .3 Results and evaluations.
43 CONCLUSION AND FUTURE WORKS. 48 LIST OF TABLES Table 1 Some examples of questions in the initial dataset. 19 Table 2 Information about question length. 20 Table 3 Some examples of short and long questions.
21 Table 4 Special questions. 24 Table 5 Data collection campaigns information. 32 Table 6 Information about data collected through campaigns. 33 Table 7 Model’s hyperparameter.
40 Table 8 Confusion matrix. 42 Table 9 Evaluation results of experimented models. 43 Table 10 Size and average prediction time. 44 LIST OF FIGURES Figure 1 Approaches to the QA problem.
2 Figure 2 An example of a decision tree. 7 Figure 3 A biological neuron. 10 Figure 4 An artificial neuron. 10 Figure 5 The structure of the RNN network.
11 Figure 6 The architecture of LSTM. 12 Figure 7 BERT, OpenAI and ELMo. 13 Figure 8 Masked LM. 13 Figure 9 An example of the input in the BERT model.
14 Figure 10 One-word CBOW model structure. 16 Figure 11 Data building process. 26 Figure 12 Written data collection process. 27 Figure 13 The data collection interface.
28 Figure 14 Training data evaluation process. 28 Figure 15 Speech data collection process. 30 Figure 16 Speech data collection interface. 31 Figure 17 Distribution of the number of words in a sentence with the Written Collection System.
34 Figure 18 Distribution of word count in the data collected by the speech system. 34 Figure 19 Text classification model architecture. 37 Figure 20 Similarity comparison model Sbert. 38 LIST OF ACRONYMS ASR Automatic Speech Recognition QA Question Answering FAQ Frequently Asked Questions NLP Natural Language Processing SVM Support Vector Machine LSTM Long-short Term Memory INTRODUCTION In this chapter, the thesis presents the reasons for choosing the topic, based on the analysis of actual needs as well as previous studies on the question-answering system in Vietnamese and in the world.
Along with that, this chapter also gives the aim, scope of the topic, research orientation, and layout of the thesis. Problem Formula Finding information is becoming more and more challenging as the amount of knowledge on the Internet is getting bigger and bigger. Conventional search engines only return lists of short paragraphs or related links, which makes it difficult for users, especially those who lack experience and search skills. Therefore, it is essential to build a question-answering system capable of giving an accurate answer to a question quickly.
Question Answering (QA) is a large branch in the field of natural language processing (NLP), which takes as input a question in the form of a natural language question, possibly text. or sound, then give the corresponding answer [1]. Classification of the QA system There are many ways to classify QA systems. Based on the data source, we can divide the QA problem into three main categories: structured data, semi- structured data, and unstructured data [2].
A knowledge graph is a representation of structured data. Semi-structured data is usually presented in the form of lists or tables. And unstructured data is often represented as text in natural language such as sentences, paragraphs, documents, etc. Based on the domain, the question- answering system is divided into two main types: open-domain QA system and closed-domain QA system [2].
The goal of an open-domain system is to answer questions in many different fields, based on data mining from rich information sources such as Wikipedia, Web Search,. Meanwhile, a closed-domain system is geared towards answering a question for a particular domain. The number of questions in a closed domain system is smaller with limited resources and participatory construction by a team of experienced experts in that field. Approaches for solving QA problems Previous studies have solved the QA problem in many different ways.
According to our knowledge, the approaches to this problem can be divided into 4 main groups as described in Figure 1 [3]. 1 Figure 1 Approaches to the QA problem. Figure 1 describes the approaches that previous studies have used to solve the QA problem, including (i) the traditional approach, (ii) Information Retrieval (IR) combined with Machine Reading Comprehension (MRC), (iii) using knowledge-based (KB) and (iv) based on similar questions with Question Entailment (QE) [3]. With the first approach, the QA problem is solved by a pipeline consisting of three main components: Question Processing, Document Retrieval, and Answer Extraction [1].
First, the user's question will be analyzed and processed by the Question Processing component. The task of this component is to understand the user's question and generate the query as input for the next component. At the same time, this component also exploits the content of the question to be able to provide useful information such as question type, entities, and important information, helping to increase the accuracy of the answer extraction process [4]. An 8-step pipeline in this module includes entity labeling, POS tagging, linguistic trimming heuristics, dependency parsing, sentiment analysis, and generating patterns for queries with ranking given by Usbeck, Ngomo, Bühmann, and Unger introduced in their study [5].
After the question has been analyzed by the first component, the Document Retrieval component will rely on that analysis to search for related documents, usually texts or paragraphs based on an IR module or Web Search 2 Engines [6]. Finally, the Answer Processing component will search and return the final answer based on those documents. To extract the answers, the research is usually based on the extraction of real information available in the documents [7] [8] [9], combined with previously analyzed answer-type information. Deploying a QA system in this approach helps to control the system in a better way, but this is a rather complicated task because it requires a combination of many natural language processing and information retrieval technologies.
The development of deep learning technologies has allowed data processing with a large amount of computation, which makes the research directions of QA problems based on reading comprehension more widely studied [1]. MRC is the problem of finding an answer to a question in natural language, based on a given passage. This passage will be selected among many text fragments in the database under the evaluation of the IR component, using document querying technologies. To solve the MRC problem, based on the answers, there are two main research directions: (I) generating the answer (Generative MRC) and (ii) extracting the answer from the passage (Extractive MRC).
In the first direction, the answer will be generated automatically based on the input information. This is also how people read and understand the content of the passage and give their answers, so the answer will be more natural and close. However, this also makes the construction of the training data more difficult and the quality assessment of the model more complicated.