VIETNAM NATIONAL UNIVERSITY UNIVERSITY OF INFORMATION TECHNOLOGY ADVANCED PROGRAM IN INFORMATION SYSTEMS NGUYEN THI NGA SENTIMENT ANALYSIS OF USER’S VIETNAMESE MOVIE REVIEWS USING SUPPORT VECTOR MACHINE BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS HO CHI MINH CITY, 2021 VIETNAM NATIONAL UNIVERSITY HCM CITY UNIVERSITY OF INFORMATION TECHNOLOGY ADVANCED PROGRAM IN INFORMATION SYSTEMS NGUYEN THI NGA - 16520787 SENTIMENT ANALYSIS OF USER’S VIETNAMESE MOVIE REVIEWS USING SUPPORT VECTOR MACHINE BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS THESIS ADVISOR Dr. CAO TH] NHAN HO CHI MINH CITY, 2021 ASSESSMENT COMMITTEE The Assessment Committee is established under the Decision. by Rector of the University of Information Technology. Bo ieceeteeeetecseseseseseeecsesesesscsessseaeseseeessseseeeaeee - Member.
ACKNOWLEDGMENTS The graduation thesis has been a golden opportunity for me to testify and implement the knowledge that I have learned during our time at the university, yet opportunities always accompany difficulties. Thus, I would like to express our special thanks of gratitude to our beloved ones that have helped me overcome this challenging period. First and foremost, we are so grateful for having Dr.Cao Thi Nhan as my thesis advisor. Were it not for her being incredibly patient, inspiring, and knowledgeable, I probably would not accomplish this thesis at this level.
Secondly, to Mr.Nguyen Minh Tri, Mr. Nguyen Van Vinh with our highest gratitude and appreciation, I am so thankful for your support. Thank you for helping me to keep track of my direction in research.I would also like to thank Pham Thi Anh Minh, Nguyen Thi Thu Huong for accompanying me in the process of creating the dataset. Last but not least, I would like to thank my parents for having been there for us as always.
In the course of doing the thesis, I could not avoid shortcomings and limitations completely. Therefore, I really look forward to receiving valuable feedback and suggestions from teachers. Once again, thank you very much! ii THESIS PROPOSAL THESIS TITLE: Sentiment Analysis of User’s Vietnamese Movie Reviews using Support Vector Machine. Cao Thi Nhan Duration: Oct 5, 2020 to Dec 15, 2020 Student: Nguyen Thi Nga — 16520787 Contents: 1.Scope e Dataset: Vietnamese movie review.
e Algorithm: SVM, N-gram, TF-IDF. e Programming language: Python. Objectives e Dataset: Vietnamese movie review. e Data preprocessing algorithms, data mining.
Methodologies ii e Survey: collect and read information from documents and textbooks related to data mining, machine learning and issues related to sentiment analysis. e Use tools to crawl data from Facebook pages about movies: CGV Cinema, Lotte,. e Research data preprocessing technologies. e Research about SVM models to use with this data.
e Evaluation: use technologies to evaluate the classification issues. Expected results e Vietnamese standard dataset of movie reviews. e Successfully build a demo. e Applying the SVM model will result in more than 60%.
Timeline: Phase 1 (5/9/2020 — 18/9/2020): Discuss to select thesis, find out the situation, related articles and thesis, write thesis outline. Phase 2 (19/9/2020 - 15/11/2020): Draw1 data, process the data and. Find out and apply solutions for abbreviations in data. Phase 3 (15/11/2020 - 8/12/2020): Install the environment and run the algorithm.
Use techniques to evaluate such as: numpy-matrix Phase 4 (8/12/2020 - 25/12/2020): Edit, supplement and complete report, slide. iv TABLE OF CONTENTS Contents ACKNOWLEDGIMMENTTS.-- GG G0001 0 1816 ii THESIS PROPOSAL.scsssseseesssssssssssessssssssesevessssseesnsseseeesesseeececssseensessesesasenseensess iii TABLE OF CONTEINTS.- --- ĂĂ 5 Họng v LIST OF TABLES .--- 5-5 nh nh nh nh ng nhevii TABLE OF FIGURES. cọ ng HH TH HH ng 0111080731777 ix ABSTRACT. T0 nhà HT v00 1001100801 10011001801 101 1 Chapter 1.
Objectives and Scope 1. Ăn ng ng ng ng 1 1111171 ng ng re 7 2.ssesssesssesssessscssscssscssscssscssscesscsnssssscenscsnscsnscssscsnscsnsssnsesnscsssssnassnscsnenenanens 10 CHAPTER 3: DATASET CREATION. Collecting and Preprocessing Data. 23 CHAPTER 4: SENTIMENT ANALYSIS MODEL.
Support Vector IVIqChrie.-- -- - «<< 5< + SE EE*EE*EE E155 3 181. VeCtOFÍZGtÍON.csscsssscssscssscssensssssssssnsennsesscsssnsssnssssesscnssensecssessanenaseeasessanensaeseasens 29 CHAPTER 5: EXPERIMENT AND RESULTS 5. VectOFÍZGEÍOTI. Implement And Setting Parameter.
Traditional Classification MOGEI. Model evaluation: numpy-IT(GEFÍX.-- c2 1n HH ng ng ng ni 51 6. Future Work REFERENCES. LIST OF TABLES Table 3.
Consensus among labeling members.2 The number and proportion of label categories. Feature unigrams, bigrams, trigrams for sentence “phim vừa hay vừa cam động phải dùng khăn giấy để lao LON”. The results for the issues of determining sentiment analysis in units percent. 234 TABLE OF FIGURES Figure 5.
1 Diagram Sentiment Analysis system overview. 2 The accuracy comparison diagram among features. 3 The Homepage of website. 4 The Contact page of website Figure 5.
5 The Demo page of website Figure 5. 6 The recommend sentence feature Figure 5. 7 The results demo of sentence “phimm hay qua” Figure 5. 8 The results demo of sentence “công vinh đá đẹp ghê”.
0 The results demo of sentence “bộ phim không được hap dân người Figure 5. 2 The results demo of sentence “góc nhìn từ cửa sô that là đẹp” Figure 5. 3 The results demo of sentence “phim cũng 6n áp! anh nhé?”. 4 The results demo of sentence “phim có vẻ không oke ti nao” Figure 5.
5 The results demo of sentence “diễn viên chính xấu ghê mà diễn cũng do nữa nhưng nhìn chung phim có nội dung hay nên xem”. 6 The results demo of sentence “trời lạnh buồn ngủ quá”. viii ABBREVIATIONS ID Acronyms Mean 1 NLP Natural language processing 2 SA Sentiment analysis 3 SVM Support Vector Machine ix ABSTRACT Sentiment analysis (or opinion mining) is a natural language processing technique used to determine whether data is positive, negative or neutral. Sentiment analysis is often performed on textual data to help businesses monitor brand and product sentiment in customer feedback, and understand customer needs.
In this document, we focus on building a sentence-level Vietnamese language standard for user comments on the data movies domain. Our set of documents includes sentences that solve sentiment analysis issues. We provide a set of rules to cater to community research and language development. The authors have planned to implement the topic as follows: e Create a dataset about reviews movies.
e Create the guidelines for the dataset. e Research and choose the model to solve the Sentiment Analysis of the movies dataset. e Evaluation the model. e Build a website to predict the sentences.
In addition, in this topic, we research and test with the Support Vector Machine model on the dataset we build. Test results for the issues of detecting sentiment the type SVM model achieved 92. However this thesis has many limits bellow: e Some words still interfere with the model with keyword such as : “không được, kha, gu, có vẻ”. e Besides the results, there are still some limitations in our thesis.
Our dataset builds unequally between labels and influences the test results. This limitation will have a great impact on bringing the application into practice. e The dataset has 5.194 rows to solve sentiment analysis issues. In this, positive has 544 sentences, the neutral has 4.326 sentences and the negative has 324 sentences.
With the distribution ratio having a large difference between the labels, it will cause linguistic imbalance and affect the results during later testing. The model is overfitted because the test dataset is overfitted with train dataset. Introduction In recent years, with the strong and rapid growth of the Internet and the need to consult the feedback of previous customers as the demand for entertainment of young people is increasing. Therefore, websites are now being developed to allow users to share experiences, reviews, comments and feedback on different types of services and movies from cinemas.
When users decide to go to a certain movie, they not only consider information about the actor, trailer, and director, but also tend to be interested in the feedback of other users. When reviewing the reviews and feedback of other users, customers tend to make decisions on choosing a more suitable and reliable movie. Along with that, businesses, services and organizations also collect feedback from users about their movies to give better directions. However, with a large amount of feedback from users about movies, it is difficult for users and businesses and organizations to care about them.
To solve these issues, businesses, organizations and users need a system that can automatically analyze all reviews and summarize all the feedback for customers and businesses to refer. and make quick decisions. Currently, the information that systems are used to analyze user feedback on websites is usually only interested in the scores that users rate about those products and services. However, the feedback rating scales do not objectively express the level of user satisfaction with sentences and comment paragraphs.
The Sentiment Analysis issues, particularly the Sentiment Analysis issues on movies dataset in the movie data domain, is very attractive to the research community in the world and in the country. Most linguistic sets and algorithms are built and experiment in many different languages such as English, Chinese, etc. However, for Vietnamese, not many linguistic sets have been built to serve the research community. Therefore, we decide to build a standard sentence-level linguistic dataset for Vietnamese to serve this issue and install a system using SVM to automatically analyze the comment.
Consequently, a number of systems have also been built to analyze user comments. But, there are no systems analysts in the movies on Vietnamese language. Objectives and Scope 1. Objectives The most important audience in this thesis is user reviews.
These reviews are exploited from user feedback on CGV Vietnamese fanpage about movies. This is the basis for building and developing datasets for the issues in this thesis. In recent years, with the strong and rapid growth of the Internet and the need to consult the feedback of previous customers as the demand for entertainment of young people is increasing. Therefore, websites are now being developed to allow users to share experiences, reviews, comments and feedback on different types of services and movies from cinemas.
When users decide to go to a certain movie, they not only consider information about the actor, trailer, and director, but also tend to be interested in the feedback of other users. In this thesis, we focus on researching and implementing traditional machine learning model Support Vector Machine and n-grams to solve the Sentiment Analysis issues on movies dataset. Scope The scope we research in this thesis is the user's review on CGV facebook about the movies. The thesis is at sentence level.
For our thesis, we perform 3 labels for sentiment analysis: negative, positive and neutral. Goals The scope we research in this thesis is the user's review on CGV facebook about the movies. e The thesis is at sentence level. e The algorithm is used: SVM, n-grams In this thesis, we research, study and solve four main goals as follows: e@ We set a standard of building a target of Vietnamese language on the sentence level for domain data movies and solving the issues of sentiment analysis.
e Build the dataset at the sentence level for the dataset movie. e Preprocessing data and data mining in movies domain. e@ We implement, test, and compare different approaches to solving issues based on traditional SVM machine learning and n-grams feature extraction. Results From the researches in the thesis, I have achieved the following results: e Building a standard Vietnamese dataset with 5.194 sentences labeled for the issues of SA.
In which, there are 324 sentences with negative sentences, positive is 544 sentences and neutral is 4. Then, we divide the data into 2: test dataset and training dataset with the corresponding ratio of 70-30 for research purposes. e Build a guideline for the labeling process. e Applying the SVM model will result in more than 60%.
We achieved results with the SVM method and 2-grams of 92. Thesis Structure This thesis is divided into 6 chapters as follows: e Chapter 1: Introduction. Thesis chapter presents reasons for choosing the thesis, our objectives, scope, our contributions in this attempt, and the issues analysis ment are also written in part.