VIETNAM NATIONAL UNIVERSITY, HANOI INTERNATIONAL SCHOOL GRADUATION PROJECT IMPROVING SALES FORECASTING MODELS BY INTEGRATING CUSTOMERS’ FEEDBACKS: A CASE STUDY OF FASHION PRODUCTS LUONG THUY VY Hanoi - Year 2023 VIETNAM NATIONAL UNIVERSITY, HANOI INTERNATIONAL SCHOOL GRADUATION PROJECT IMPROVING SALES FORECASTING MODELS BY INTEGRATING CUSTOMERS’ FEEDBACKS: A CASE STUDY OF FASHION PRODUCTS SUPERVISOR: Faculty of Applied Science, Assoc. Tran Thi Oanh (Academic title, academic degree, full name) STUDENT: Luong Thuy Vy STUDENT ID: 20071004 COHORT: Computer Science MAJOR: Business Data Analytics Hanoi - Year 2023 2 ACKNOWLEDGEMENTS First and foremost, I would want to express my sincere gratitude to Assoc. Tran Thi Oanh, who served as my graduation thesis mentor, for her advice during the program. I believe that I would have a very difficult time finishing my report without her good direction.
This subject is complicated as it has been developed from “Project” subject in the last year of my program known as “Ecommerce Analytics” problem; hence it is vital to be thankful of one fintech company named as Ciaolink in Ha Noi leaded by CEO Nguyen Trung Hieu, who acted as a client of the initial problem to urge me to work hard about this up-to-date subject with a variety of new ideas, along with tough requirements consuming lots of time, resources, human labor to have experiments, to check results compared to hypotheses supposed that putting semantic features into forecasting models would make error rate diminished, meaning reactions play an vital role in ecommerce analytics in order to bring about merits to both enterprises and customers nowadays. More importantly, since this graduation thesis project is also my hard-working research I would like to send my gratitude to Directors of Ministry of Science and Technology in general, Institute for Patent and Technology Exploitation in particular gave me immediate support in tough situation, Mr. Nguyen Trong Nghia, worked as an AI engineer in IHOUZZ company, who has main contribution in initial solution of this problem to help me develop more so that I would be able complete my subject in graduation thesis. Without his support and his broad knowledge in most of fields are involved in Information Technology and overworked attitude of us, it would be definitely tough for me to overcome myself and complete all of my works.
3 PLEDGE My interest in pursuing a career in I hereby declare that this is my own work and that it is being reviewed by Assoc. Tran Thi Oanh. This graduation project most definitely honest, reliable and followed the data privacy policy of Shopee. The research findings on this subject are accurate and have not been released in any forms.
For analysis, commentary, and review, the author collected official papers, and images from a variety of sources, website completely citing each one in the reference section. This report is truthful; if fraud is discovered, I will be held accountable for the accuracy of my thesis. 4 LIST OF ACRONYMS ARIMA Autoregressive Integrated Moving Average BERT Bidirectional Encoder Representations for Transformers BPNN Back-Propagation Neural Networks CNN Convolutional Neural Network EM Expectation Maximization LSSVR Least Square Support Vector Regression LSSVRTS Least Square Support Vector Regression with Time Series LSTM Long Short - Term Memory ML Machine Learning MSE Mean Square Error MAE Mean Absolute Error NLP Natural Language Processing NARNN Nonlinear Autoregressive Neural Network PhoBERT “Phở” Bidirectional Encoder Representations for Transformers RNN Recurrent Neural Networks RoBERTa Robustly Optimized BERT Pre-training Approach MAE Mean Absolute Error SARIMA Seasonal Autoregressive Integrated Moving Average SentiWordNet A lexical resource explicitly devised for supporting sentiment classification and opinion mining applications 5 SVM Support Vector Machine viBert4news One of monolingual BERT-based models, BERT for Vietnamese is trained on more 20 GB news dataset viBERT FPT Vietnamese BERT pre-trained model of FPT.AI ViELECTRA FPT pre-trained model for Vietnamese ELECTRA VLSP Vietnamese Language and Speech Processing VPS Virtual Private Server XGBoost Extreme Gradient Boosting XLMR XLM-RoBERTa, Unsupervised Cross- lingual Representation Learning at Scale 6 LIST OF FIGURES Figure 1. Example of an item' interface on Shopee.
Rating with context actions. Relationship between Total sold with Total like. Relationship between Total historical sold with Total rating with context 18 Figure 3. A framework for integrating customers' feedback into forecasting models.
Data Collection Process. Self-labeled comment on shopee. Self-labeled comment on shopee exclude time. Comment data sample crawled manually.
Total number of comments on Shopee platform divided into 3 labels. Clean text function in sentiment analysis. Cleaned text sample. Comment extraction function.
Example of cleaned data with predicted sentiment label. Value counts of each label in training set. Bar plot show value counts of each label in train set. Bar plot show value counts of each label in dev set.
Bar plot show distribution of the length of sentences in train set. Frequency of appearance of each word in the train set. Frequency of appearance of each word in the dev set. Frequency changes across items.
52 7 LIST OF TABLES Table 4. Data crawled statistics in initial progress. Data crawled statistics in updated progress. Results of PhoBERT model on each label (%).
Results of LSTM model on each label (%). Results of CNN model on each label (%). Result of three models on sentiment analysis. Results of sales prediction models with/without integrated with customers' feedback by using 1 model built to predict 300 items.
56 8 TABLE OF CONTENTS Contents ACKNOWLEDGEMENTS. 4 LIST OF ACRONYMS. 5 LIST OF FIGURES. 7 LIST OF TABLES.
8 TABLE OF CONTENTS. Background of the study. Structure of the study. About the Project.
21 CHAPTER 2: LITERATURE REVIEW. 27 CHAPTER 3: RESEARCH METHODOLOGY. Sentiment Analysis Models. Sales Forecasting Models.
Integrating reviews and ratings of customers into forecasting models. 35 CHAPTER 4: RESULTS ANALYSIS AND DISCUSSIONS. Data Pre-processing. Exploratory Data Analysis (EDA).
EDA comment data to train PhoBERT. EDA data to load in sales forecasting models. Experiment results of sentiment analysis models. Experiment results of sales forecasting models.
56 CHAPTER 5: CONCLUSION AND FUTURE RESEARCH. Discussions and conclusion. Contribution and implications of study. Recommendation & future research of the study.
67 10 ABSTRACT In this project, I investigate the task of predicting sales for fashion companies – an interesting yet unexplored sector in e-commerce areas - by utilizing advanced machine learning models with rich features. This can help businesses predict sales by using data from past transactions and other indicators. Moreover, we also examine the influence of customers’ online feedbacks (i., consumers’ ratings and comments) on the performance of the sale forecasting models. To verify the effectiveness of integrating customers’ feedbacks into the sale forecasting models, we manually collect and annotate a dataset on the Shopee platform to conduct extensive experiments.
Experimental results on this dataset showed that integrating this kind of information indeed boosts the sale forecasting models’ accuracy significantly in evaluation metrics such as MAE and RMSE scores. Background of the study Potential consumers commonly examine online customers’ reviews before deciding buying the products. There are two types of reaction having in Shopee platform known as customer’ reviews and rating. While customer’ reviews are feedback of customers in text, icon which express their feelings after receiving package, rating of item is considered to make before purchasing goods, meaning there is a variety of ways depend of seller of each shop to hack this unreasonable reaction.
Moreover, in case of fetching comments in my research, there are a large amount of comment crawled null and only has rating score since at time tool crawler processed, customers could remove that comment. Therefore, it is essential to remove rating feature in integrating with sentiment and daily quantity sold for sales forecasting in order to decrease noisy data. Sales of products are greatly influenced by online reviews. It may reduce or increase their uncertainty about the product by providing customers with rich information about the users’ experience with products.
The growth of Internet technology has led to an increase in the popularity of online reviews. Many Vietnamese e-commerce companies, like Shopee, Lazada, and Sendo, have set up online review platforms to entice customers to publish feedback on goods in exchange for prizes. Customers' behavior patterns vary as a result, which has an impact on their purchasing decisions. Potential customers can read these reviews to guide their purchasing selections because they frequently indicate personal feelings about products, such as negative, neutral, or favorable attitudes.
Therefore, methods for measuring sentiments expressed through the content of online critiques have been developed. Research objectives Sales forecasting is the process of estimating future revenue by predicting the amount of product or services a company will sell in the future. If the companies know in product sales forecasting in advance, they are capable of coming up with a solid strategy for marketing, sales management, production, and other areas to increase their profits and lower their costs [1]. According to many research, one of the primary factors 12 which influences consumers' purchasing decisions is customers’ feedbacks.
Therefore, they may reduce or increase their uncertainty about the product by providing customers with rich information about the users’ experience with products. In this work, a method using the convolutional neural network (CNN) model for full memory sales forecasting with the results of the PhoBERT model is suggested to forecast product sales using data from product reviews. This approach uses the pre-trained BERT model to analyze the sentiment of online comments and then applies the sentiment to increase the precision of the sale forecasting models. We are aware of very few studies that have included internet review content in an effort to increase product sales.
Furthermore, the suggested method's forecasting effectiveness is assessed in this study using real-world fashion data from a Vietnamese platform called Shopee. Machine learning, which is a large branch of the tree, has several specializations, including data mining, artificial intelligence, virtual and real-time interaction, and prediction. We will solely be concentrating on machine learning-based prediction in this study. The topic that we choose in this project is data analysis on the Shopee e-commerce site.
The purpose of this project is to document the work we have done to come up with solutions to problems, thereby making predictions for the trend of women's fashion items on Shopee. In detail, we monitor a group of products in Fashion sector named “Jacket, Coat, Vest” on Shopee platform. There are some good features on Shopee that we would use as the input for this research. For each product, at a time there will be information: o Product information (Item, ID, name, Link) o Number of reviews (rating, comments) o Total of comments for each item o User name who left reviews for each item o Comment time (Y-mm-dd H:M:S) o Rating o Quantity sales in continuous days o Price 13 Thereby building related charts, giving growth forecasting models of Products, “Jacket, Coat, Vest” Group, Shops, Items.
Then, we would make comparisons of models built to choose model has the least error with a view to predicting quantity sales daily in Fashion industry. We especially focus on models for sentiment analysis and models for time series data, enhancing results of previous studies using phenomenal models in Machine Learning. This research also helps sellers on the Shopee e-commerce platform to refer to trends of women's fashion in the near future. Hopefully, our analytical models will help salespeople optimize performance and finance, capture the right customer psychology, in order to create the highest profit and revenue for the business.
As a result, businesses can confidently compete with other competitors. It is true that if the e- commerce platform could forecast its sales for the upcoming month or day, it would be able to make better business decisions. They will also be able to spot trends in their sales if a festival or event happens annually.