VIETNAM NATIONAL UNIVERSITY, HANOI INTERNATIONAL SCHOOL GRADUATION PROJECT DEVELOPING AN INTELLIGENT PDF CHATBOT USING OPENAI SUPERVISOR: PhD. Truong Cong Doan STUDENT: Pham Trong Khiem STUDENT ID: 19071049 COHORT: MIS2019A MAJOR: Management Information System Hanoi – 2024 DECLARATION OF ORIGINALITY I affirm that, to the best of my understanding, the material presented in this thesis is the result of my own efforts, conducted under the supervision of Mr. Truong Cong Doan. This thesis has not been presented for any other degree or purpose.
I confirm that the intellectual content contained in this thesis is a result of my independent work, and I have duly acknowledged any assistance received and appropriately cited all sources during its preparation. Khiem Pham Trong Khiem 1 ACKNOWLEDGEMENT To commence, I wish to extend my heartfelt appreciation to the faculty at the International School - Vietnam National University, Hanoi. I am sincerely thankful to Mr. Truong Cong Doan for his unwavering support and encouragement during the completion of my graduation thesis.
His guidance not only provided me with a different perspective on this thesis but also imparted additional knowledge in the field. It is through his assistance that I was able to successfully complete this thesis. Additionally, I want to express special gratitude to my former colleagues at Online School Education Technology JSC and my current colleagues at Viettel Digital Services. Their collaboration has significantly contributed to my understanding and research, adding substantial value to the thesis.
Last but certainly not least, I appreciate the support of my friends and family who stood by me throughout the entire thesis process. Your timely sharing, provision of time and space, and mental encouragement were crucial in allowing me to conduct my research with peace of mind. Once again, thank you all for helping me become the person I am today. Sincerely, Pham Trong Khiem 2 List of Abbreviations PDF Portable Format Document AI Artificial Intelligent NLP Natural Language Processing GPT Generative Pre-training Transformer API Application Programming Interface LLM Large Language Model IVR Interactive Voice Response UX/UI User Experience/ User Interface Q&A Question and Answer ML Machine Learning CPU Central Processing Unit GPU Graphics Processing Unit 3 List of Figures Figure 2.1: Demonstration of Chatbot.2: Demonstration of Chatbot benefits .3 The flowchart of natural language processing (NLP).1: How a AI-based Chatbot work .2: How a Rule-based Chatbot works .3: Position of Hybrid Chatbot .4: Demonstrate detailed about LangChain framework .5 Comparison between different model of Chat GPT .6: System Flow of PDF Chatbot.7: Compile on the API pricing of different model .8: Ranking MTEB (Massive Text Embedding Benchmark) of HuggingFace .9: Chat with PDF Chatbot .10: Chat with PDF Chatbot 2 .11: Syntax to run the HuggingFace Embeddings .12: Text file include to download related packages.13: The Chatbot PDF Interface .14: Right side section of the Chatbot .1: Download requirements file .2: The Requirements file .3: Checking the Confusion rate of the Chatbot .4: The Chatbot reply without Confusion .5: A PDF Chatbot run in GoogleColab.6: A multi-functional PDF Chatbot interface .7: Requirement for Installation and Setting of a high quality PDF Chatbot.
44 4 TABLE OF CONTENTS ABSTRACT .4 Scope and Limitations .5 Significance of the Study. LITERATURE REVIEW AND THEORITICAL FRAMWORK .1 Overview of Chatbots .2 Evolution of Chatbot Technologies .3 Applications of Chatbots .4 Challenges in Chatbot Development .5 Artificial Intelligent in Chatbot Development .1 What is a Chatbot? .2 Types of Chatbot .4 AI chatbot benefits .5 AI chatbot challenges .6 Natural Language Processing .1 What is NLP? .2 Why is NLP important? .3 How does NLP work? .7 Relevance to PDF Chatbot Development. Navigating Interactive Documents:. Effective Information Acquisition:.
Help & Support for Users: .4 Optimization of Workflow:. RESEARCH DESIGN AND IMPLEMENTATION .1 Programming Languages and Tools.2 Choice of OpenAI Model .4 Integration of OpenAI API .5 User Interface Design .1 Metrics for Chatbot Performance .3 Comparison with Existing Solutions .5 Limitations of the Study .1 Recap of Objectives .2 Contributions and Implications of the Study .3 Recommendations for Future Research. 48 6 ABSTRACT In the contemporary era, society is deeply immersed in technology, and the rapid progress of Artificial Intelligence (AI) has significantly impacted diverse industries, paving the way for novel applications. A prominent domain where AI prominently demonstrates its capabilities is in chatbot technology.
Initially conceived as a computer program to simulate human conversation, chatbots have evolved into sophisticated and intelligent entities, thanks to the transformative influence of AI. This thesis endeavors to delve into the intricate process of developing a chatbot using OpenAI, aiming to create an intelligent tool that enhances PDF document search and retrieval through natural language processing and machine learning. The study not only focuses on the practical development aspects but also delves into the theoretical foundations of AI and Natural Language Processing (NLP). The research comprehensively details the planning, design architecture, and developmental processes involved in creating an AI-driven chatbot specifically tailored for PDF document interaction.
An exhaustive evaluation method is employed, utilizing metrics to assess chatbot performance across various categories. Additionally, user feedback and a comparative analysis with existing products contribute valuable insights to enhance the chatbot's effectiveness. The thesis also highlights technical challenges and limitations encountered during the study, aiming to provide clarity and guide future endeavors for improvement. It is crucial to note that this thesis serves as a modest solution, focusing on augmenting user interactions with PDF documents to expedite access.
It reflects just a fragment of AI's potential. The outcomes of this research aspire to lay the groundwork for future advancements in AI-driven conversational technology, offering faster and more efficient solutions for various applications.1 Background In the ever-evolving landscape of artificial intelligence (AI), the integration of sophisticated technologies has significantly transformed human-machine interactions. One notable development within this realm is the evolution of chatbots, which leverage NLP to facilitate seamless communication between users and machines. Despite these advancements, the ubiquitous use of Portable Document Format (PDF) documents as a medium for information exchange remains a prevalent aspect of contemporary communication.
This thesis delves into the convergence of AI, document processing, and user interaction by concentrating on the creation of an Intelligent PDF Chatbot using OpenAI technologies. The primary aim is to harness the capabilities of OpenAI's advanced language model, GPT (Generative Pre-trained Transformer), to elevate the functionalities of a chatbot embedded within the context of PDF documents. This research seeks to address the intricate challenges associated with extracting, comprehending, and responding to textual content within PDFs, contributing to the progression of intelligent conversational agents specifically tailored for document-based interactions. Through a comprehensive exploration that encompasses theoretical foundations, system design considerations, and empirical evaluations, this thesis aspires to offer valuable insights into the potential advantages, limitations, and future prospects of deploying intelligent PDF chatbots in practical, real-world scenarios.2 Motivation This thesis is propelled by the imperative to revolutionize the way individuals engage with digital content, particularly within the prevalent landscape of Portable Document Format (PDF) documents.
As AI continues to redefine the boundaries of technology, the integration of sophisticated conversational agents, such as chatbots powered by OpenAI's GPT, emerges as a transformative avenue for enhancing user experiences. The motivation lies in recognizing the enduring significance of PDFs as a primary medium for information dissemination and the untapped potential to infuse intelligence seamlessly into this document format. By delving into the development 8 of an Intelligent PDF Chatbot using OpenAI technologies, this research aims to transcend the conventional boundaries of static document interaction. The aspiration is to create a dynamic and intelligent interface that not only facilitates efficient information retrieval but also navigates the nuances of natural language within PDF contexts.
This thesis is driven by the desire to contribute to the evolution of document-centric AI applications, offering a more intuitive, responsive, and user- friendly means of accessing and interacting with information in the ever-evolving digital landscape.3 Objectives The objectives of this thesis are multifaceted, encompassing the comprehensive development of an Intelligent PDF Chatbot leveraging OpenAI technologies. Firstly, the study aims to formulate advanced algorithms for proficient document parsing, content extraction, and contextual comprehension within diverse PDF structures. The integration of OpenAI's GPT, a pivotal aspect, is targeted to harness its natural language processing capabilities, enhancing the chatbot's capacity for generating contextually relevant and coherent responses within the PDF environment. User- centric design principles guide the development of an intuitive interface to optimize user experiences, facilitating seamless interactions.
Dynamic context handling mechanisms are implemented to enable the chatbot to sustain coherent conversations across multiple user queries within a single PDF document. Additionally, the thesis endeavors to extend the chatbot's capabilities for intelligent interactions with multiple PDF documents simultaneously, offering users a more versatile and efficient information retrieval process. Rigorous performance optimization, a key objective, ensures the chatbot's efficiency and responsiveness, while a training mechanism based on machine learning techniques enables continuous adaptation and improvement. The establishment of a robust evaluation framework aims to assess the chatbot's accuracy, performance, and usability within the PDF context.
The system's adaptability to diverse document formats, integration of security measures, scalability, and a comparative analysis with existing solutions underscore the holistic approach of this research. Ultimately, thorough documentation facilitates knowledge 9 transfer, providing a foundation for future enhancements or adaptations in the dynamic landscape of intelligent PDF-based conversational agents.4 Scope and Limitations The scope of this thesis is dedicated to the comprehensive development and application of an Intelligent PDF Chatbot utilizing OpenAI technologies, with a primary focus on elevating user interactions within the domain of PDF documents. This research aims to navigate the intricate landscape of natural language processing and document parsing, with a specific emphasis on constructing a sophisticated system capable of intelligently comprehending and responding to diverse user queries embedded within the unique structures of PDF documents. The project's ambit extends beyond mere user engagement to explore the chatbot's adaptability across a spectrum of document formats, allowing for nuanced interactions with multiple PDFs concurrently.
The envisioned scope, however, invites consideration of certain limitations. The effectiveness of the chatbot might hinge on the quality and consistency of document formatting, presenting challenges in scenarios involving highly unstructured or irregularly formatted PDFs. Furthermore, while the chatbot aspires to handle a wide array of user queries, its performance may encounter constraints associated with the complexity of certain linguistic nuances and variations in user input styles. Additionally, despite substantial efforts to address security and privacy concerns within the PDF context, the inherent challenges of handling sensitive information pose considerations for further refinement.
Acknowledging these constraints, the thesis endeavors to provide a nuanced and insightful exploration, offering a comprehensive understanding of both the capabilities and potential challenges associated with the development of an Intelligent PDF Chatbot using OpenAI technologies.5 Significance of the Study This study on the development of an Intelligent PDF Chatbot using OpenAI technologies holds profound significance as it endeavors to redefine the conventional dynamics of user interaction within the prevalent realm of PDF documents. Recognizing the enduring prominence of Portable Document Format (PDF) in 10 information dissemination and archival practices, this research seeks to imbue AI, particularly harnessing OpenAI's GPT, into the very fabric of PDF interactions. The study's pivotal importance lies in its ambitious goal to seamlessly blend static document content with dynamic, user-driven conversations. If successful, the Intelligent PDF Chatbot stands to revolutionize the user experience, offering an unprecedented fusion of intuition, efficiency, and interactivity in navigating and extracting information from PDFs.
Beyond its immediate application, this research contributes to the cutting-edge landscape of AI-driven conversational agents, pushing the boundaries of their adaptability to the intricate structures inherent in PDF documents. The profound implications of this study extend to shaping the future landscape of human-computer interactions, not merely confined to the PDF domain but also influencing the broader spectrum of AI applications in information-rich environments. In essence, the significance of this study transcends its immediate objectives, holding the promise of ushering in a new era of intelligent and responsive interactions with digital documents. LITERATURE REVIEW AND THEORITICAL FRAMWORK 2.