A Vietnamese Text-based Conversational Agent Nguyen Quoc Dai Faculty of Information Technology University of Engineering and Technology Vietnam National University, Hanoi Supervised by Dr. Pham Bao Son A thesis submitted in fulfillment of the requirements for the degree of Master of Science in Computer Science November 2011 LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com ORIGINALITY STATEMENT ‘I hereby declare that this submission is my own work and to the best of my knowledge it contains no materials previously published or written by another person, or substan- tial proportions of material which have been accepted for the award of any other degree or diploma at University of Engineering and Technology (UET/Coltech) or any other educational institution, except where due acknowledgement is made in the thesis. Any contribution made to the research by others, with whom I have worked at UET/Coltech or elsewhere, is explicitly acknowledged in the thesis. I also declare that the intellectual content of this thesis is the product of my own work, except to the extent that assistance from others in the project’s design and conception or in style, presentation and linguistic expression is acknowledged.’ Hanoi, November 23rd , 2011 Signed.
i LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com ABSTRACT The first step that a question answering system must perform is to transform an input question into an intermediate representation. Most published works so far use rule-based approaches to realize this transformation in question answering sys- tems. Nevertheless, in existing rule-based approaches, manually creating the rules is error-prone and expensive in time and effort. In this thesis, we focus on introduc- ing a rule-based approach that offers an intuitive way to create compact rules for extracting intermediate representation of input questions.
Experimental results are promising where our system achieves reasonable performance and demonstrate that it is straightforward to adapt to new domains and languages. More importantly, this thesis introduces a Vietnamese text-based conversational agent architecture on specific knowledge domain which is integrated in a question answer- ing system. When the question answering system fails to provide answers to user input, our conversational agent can step in to interact with users to provide answers to users. Experimental results are promising where our Vietnamese text-based con- versational agent achieves positive feedback in a study conducted in the university academic regulation domain.
Publications: ? Dai Quoc Nguyen, Dat Quoc Nguyen and Son Bao Pham. A Vietnamese Text-based Conver- sational Agent. of The 25th International Conference on Industrial, Engineering & Other Applications of Applied Intelligent Systems (IEA/AIE 2012), Springer-Verlag LNAI, pp. ? Dai Quoc Nguyen, Dat Quoc Nguyen and Son Bao Pham.
A Semantic Approach for Ques- tion Analysis. of The 25th International Conference on Industrial, Engineering & Other Applications of Applied Intelligent Systems (IEA/AIE 2012), Springer-Verlag LNAI, pp. ? Dat Quoc Nguyen, Dai Quoc Nguyen and Son Bao Pham. Systematic Knowledge Acquisition for Question Analysis.
of the 8th International Conference on Recent Advances in Natural Language Processing (RANLP 2011), ACL Anthology, pp. ii LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com iii ? Dai Quoc Nguyen, Dat Quoc Nguyen, Khoi Trong Ma and Son Bao Pham. Automatic On- tology Construction from Vietnamese text. In Proceedings of the 7th International Conference on Natural Language Processing and Knowledge Engineering (NLPKE’11), IEEE, pp.
? Dat Quoc Nguyen, Dai Quoc Nguyen, Son Bao Pham and Dang Duc Pham. Ripple Down Rules for Part-Of-Speech Tagging. of 12th International Conference on Intelligent Text Processing and Computational Linguistics (CICLING 2011), Springer-Verlag LNCS, part I, pp. ? Dai Quoc Nguyen, Dat Quoc Nguyen and Son Bao Pham.
A Vietnamese question answering system. In Proceedings of the 2009 International Conference on Knowledge and Systems Engineer- ing (KSE 2009) , IEEE CS, pp. LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com ACKNOWLEDGEMENTS First and foremost, I would like to express my deepest gratitude to my supervisor, Dr. Pham Bao Son, for his patient guidance and continuous support throughout the years.
He always appears when I need help, and responds to queries so helpfully and promptly. I would like to give my honest appreciation to my younger brother, Nguyen Quoc Dat, for his great support. I would like to specially thank Prof. Bui The Duy and my colleagues for their help through my time at Human Machine Interaction Laboratory, UET/Coltech.
I sincerely acknowledge the Vietnam National University, Hanoi, Toshiba Founda- tion Scholarship, and especially Dr. Pham Bao Son for supporting finance to my master study. Finally, this thesis would not have been possible without the support and love of my mother and my father. Thank you! iv LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com To my family ♥ v LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com Table of Contents 1 Introduction 1 1.1 A Semantic Approach for Question Analysis .2 A Vietnamese Text-based Conversational Agent .1 Text-based conversational agents .1 Using keywords for pattern matching .2 Using the sentence similarity measure for pattern matching .2 FrameScript Scripting Language .3 Question answering systems.
12 3 Our Question Answering System Architecture 15 3.1 Vietnamese Question Answering System .1 Natural language question analysis component .1 Intermediate representation of an input question .2 Answer retrieval component .2 Using FrameScript for question analysis .2 Syntactic analysis module .3 Semantic analysis module. 22 4 Text-based Conversational Agent for Vietnamese 24 4.1 Overview of architecture .2 Determining separate contexts .3 Identifying hierarchical contexts. 27 vi LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com TABLE OF CONTENTS vii 5 Evaluation and Discussion 29 5.1 Experimental results for Vietnamese text-based conversational agent .2 Question Analysis for English. 33 6 Conclusion 34 A Scripting patterns for English question analysis 36 B Definitions of question-class types 38 C Definitions of question-structures 40 LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com List of Figures 2.1 O’Shea et al.’s conversational agent framework.1 Architecture of our question answering system.2 Architecture of the natural language question analysis component using FrameScript.1 Architecture of our Vietnamese text-based conversational agent.
25 viii LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com List of Tables 4.1 Script examples of “subjects” .2 Transformations between contexts .3 Order of transformation rules .4 Ordered transformation between contexts .1 List of transformations among contexts .3 The satisfied degree of students .4 Number of rules corresponding with each question-structure type .5 Number of rules with conditional responses .6 Number of questions corresponding with each question-structure type 32 5. 32 ix LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com List of Abbreviations CA Conversational Agent QA Question Answering IR Information Retrieval IE Information Extraction GATE General Architecture for Text Engineering JAPE Java Annotation Patterns Engine NLIDB Natural Language Interface to DataBase POS Part-of-Speech NLP Natural Language Processing GUI Graphic User Interface x LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com Chapter 1 Introduction 1.1 A Semantic Approach for Question Analysis The goal of question answering systems is to give answers to the user’s questions instead of ranked lists of related documents as used by most current search engines (Hirschman and Gaizauskas, 2001). Natural language question analysis component is the first component in any question answering systems. This component creates an intermediate representation of the input question, which is expressed in natural language, to be utilized in the rest of the system.
For the task of translating a natural language question into an explicit intermedi- ate representation of the complexity in question answering systems, most published works so far use rule-based approach to the best of our knowledge. Some question answering systems such as (Lopez et al., 2007; Phan and Nguyen, 2010) manually defined a list of sequence rule structures to analyze questions. However, in these rule-based approaches, manually creating the rules is error-prone and expensive in time and effort. In this thesis, we present an approach to return an intermediate representation of question via FrameScript scripting language (McGill et al.
Natural lan- guage questions will be transformed into intermediate representation elements which include the construction type of question, question class, keywords in question and semantic constraints between them. Framescript allows users to intuitively write rules to directly extract the output tuple. 1 LUAN VAN CHAT LUONG download : add luanvanchat@agmail.2 A Vietnamese Text-based Conversational Agent A text-based conversational agent is a program allowing the conversational inter- actions between human and machine by using natural language through text. The text-based conversational agent uses scripts organized into contexts comprising hier- archically constructed rules.
The rules consist of patterns and associated responses, where the input is matched based on patterns and the corresponding responses are sent to user as output. We focus on the analysis of input text in building a conversational agent. Re- cently, the input analysis over user’s statements have been developed following two main approaches: using keywords (ELIZA (Weizenbaum, 1983), ALICE (Wallace, 2001), ProBot (Sammut, 2001)) and using similarity measures (O’Shea et al., 2010; Graesser et al., 2004; Traum, 2006) for pattern matching. The approaches using keywords usually utilize a scripting language to match the input statements, while the other approaches measure the similarity between the statements and patterns from the agent’s scripts.
In this thesis, we introduce a Vietnamese text-based conversational agent ar- chitecture on a specific knowledge domain. Our system aims to direct the user’s statement into an appropriate context. The contexts are structured in a hierarchy of scripts consisting of rules in FrameScript language (McGill et al. In addition, our text-based conversational agent was constructed to integrate in a Vietnamese question answering system.
Our conversational agent provides not only information related to user’s statement but also provides necessary knowledge to support our question answering system when it is unable to find an answer. The knowledge domain we used to build our text-based conversational agent is the academic regulation at Vietnam National University, Hanoi (VNU). The aca- demic regulation book helps students to know the course programs, the regulation of examinations, the discipline at VNU. However, most students don’t prefer reading the academic regulation book.
Therefore, our contribution creates an interaction channel to offer the necessary information to students. Once students give their statements that they are interested in the academic regulation, our text-based con- versational agent responses these statements by providing the related information in detail. Furthermore, our conversation agent also interacts with students by offering the option to ask if students want to know other information. LUAN VAN CHAT LUONG download : add luanvanchat@agmail.3 Thesis Organisation This dissertation consists of 6 chapters.
In chapter 2, we provide some literature re- views and describe our Vietnamese question answering system architecture, in which we present a method for converting a natural language question into an intermediate representation, in chapter 3. We propose our Vietnamese text-based conversational agent architecture in chapter 4. We describe our experiments and discussions in chapter 5, and conclusion will be presented in chapter 6. LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com Chapter 2 Literature review In this chapter, we review related works using text-based approaches for conversa- tional agent (CA).1 describes the approaches constructing rules to match user’s natural language utterances in the ways of using keywords (in section 2.1) and using a sentence similarity measure (in section 2.
In addition, section 2.2 covers the basic knowledge background about FrameScript scripting language that we have been working on, while section 2.3 presents reviews about the question answering systems driving specific-domains.1 Text-based conversational agents 2.1 Using keywords for pattern matching ELIZA (Weizenbaum, 1983) was one of the earliest text-based conversational agents based on a simple pattern matching by using the identification of keywords from user’s statement. Then ELIZA transforms the user’s statement to an appropriate rule and generates output response. The procedure that ELIZA responds to an user input to give an appropriate output consists of five steps. • Identify the important keywords appearing in user’s statement.
• Define some minimal context within which selected keyword occurs. • Determine an appropriate transformation rule. • Generate the responses when the input text contained no keywords. 4 LUAN VAN CHAT LUONG download : add luanvanchat@agmail.