NATIONAL UNIVERSITY HOCHIMINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY ADVANCED PROGRAM IN INFORMATION SYSTEMS DANG TRUC LAM - 19521736 BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS THESIS ADVISOR PROF. DO PHUC MSC. NGUYEN THI KIM PHUNG HO CHI MINH CITY, 2023 ASSESSMENT COMMITTEE The Assessment Committee is established under the Decision. by Rector of the University of Information Technology.
- Member UNIVERSITY OF INFORMATION TECHNOLOGY — VIETNAM NATIONAL UNIVERSITY FACULTY OF INFORMATION SYSTEMS BIEN BAN CHINH SUA KHOA LUAN TOT NGHIEP (MINUTES OF AMENDED THESIS DEFENSE) Sinh vién (Student): Dang Truc Lam MSSV (Student ID): 19521736 Thuộc chuyên ngành (Major): Advanced program in information systems Khoa (Academic year): 2019 Thực hiện đề tai (Thesis project): Detecting sources using YouTube original videos to manipulate contents with other purposes Hôm nay, ngày 28 tháng 01 năm 2024, tôi đã hoàn tat việc chỉnh sửa Khóa luận tốt nghiệp (KLTN) theo ý kiến của Hội đồng châm KLTN và phản biện với các nội dung sau đây: Today, January 28th, 2024, I have completed the editing of my graduation thesis according to the comments from the Thesis Defense Council and Thesis Reviewer with the following content: : Trang số Trang số No. Nội dung can phải chỉnh sửa (Page Nội dung đã chỉnh sửa (Page (Details needed to be amended) (List of amended details) number) number) A Chinh sửa theo yêu cầu của phản biện (Based on the Reviewer request) Add the experimental results 49 Add more results of experiment|49, 50, 51, : section fully and clearly process 52, 53, 54 B Chinh sửa theo yêu cau của Hội đồng (Based on the Thesis Defense Council request ) The result is not clear. % is 52 Change the representation scale 52 h not suitable for distance of cosine distance Topic is too big and challenge 7 Re-define the scope and 7 objectives of the project, make > more clearer goal of each process in objectives section. Need to change the title 1,11 Change the title from “Detecting) 1,1 Sources using YouTube original videos to manipulate contents ` with other purposes” to “Propose an approach to compare the contents of YouTube videos” Dinh kèm Biên bản này là phiếu nhận xét phản biện và biên bản chấm bảo vệ KLTN (Attached to the minutes is the assessment sheet of the Thesis Reviewer and the Thesis Defense Council evaluation minutes for the graduation thesis.) Ho Chi Minh City, 28th January 2024 Xác nhận của Giảng viên phan biện Sinh viên thực hiện (Confirmation of Thesis Advisor) (Student) (Ký & ghi rõ họ tên) (Ký & ghỉ rõ họ tên) il Xác nhận của Trưởng/Phó (Khoa/Bộ môn) phụ trách (Confirmation of Head of Faculty) (Ký & ghi rõ họ tên) 11 UNIVERSTTY OF INFORMATION TECHNOLOƠY ADVANCED PROGRAM 4EẺ Ediscostiofr Advanced IN INFORMATION SYSTEMS Program COMMENTS OF THESIS’S REVIEWER Thesis title: Detecting sources using YouTube original videos to manipulate contents with other purposes Students: Reviewer: Dang Truc Lam — 19521736 Dr.
Cao Thi Nhan Comments 1. Structure and layout - Sections are quite clear and logical. - However, the experimental content has not been fully presented. The video frame analysis part has not been written yet.
Students must complete all sections of the report. Contents - The student learned about determining whether a video file has been edited or not. This is an interesting topic and still has many challenges. - The student proposed a method focusing on audio analysis and video frame analysis extracted from original video, then used cosine metric to calculate the similarity.
Based on the similarity to make the conclusion. - This approach is appropriate, but the experimental results are not fully shown in the report. Therefore, the student must add the experimental results section fully and clearly. Experiments and application iv Experiments - The ISOT dataset includes two categories of articles: fake and real news.
The dataset was acquired from authentic sources, with the truthful articles being extracted using web crawling of articles from Reuters. The fake articles were gathered from untrustworthy websites that were identified by Politifact (a reputable fact-checking group in the USA) and Wikipedia. This is a good data set. - Carried out experiment YouTube2Text using Whisper.
Behavior - The student shows good attitudes while working with the reviewer. Mark: Dang Truc Lam: 7.0/10 Ho Chi Minh city, January 22", 2023 Reviewer Cao Thi Nhan EN BAN 0110911119911 711 0/7 2)/7 COMMENTS (From the instructor) viii COMMENTS (From the thesis advisor) ix ACKNOWLEDGMENTS First of all, allow me to express my sincere appreciation to my advisors, Prof. Do Phuc and MSC. Nguyen Thi Kim Phung, for their extraordinary instruction, assistance, and knowledge during the course of this research undertaking.
Their perceptive critique, support, and steadfast dedication have played a critical role in molding this thesis and fostering my scholarly development. The guidance and clarity that Prof. Do Phuc's extensive knowledge and experience in Machine Learning have provided to my research have been invaluable. His commitment to my scholarly growth has been phenomenally motivating, and I am appreciative of the inspiration he has provided through his mentoring.
The supervision and constructive feedback provided by MSC. Nguyen Thi Kim Phung have made a substantial contribution to the enhancement of this thesis. Her erudite counsel and meticulous attention to detail were indispensable in elevating the caliber of my research endeavors. I express gratitude for her unwavering support and tolerance during this endeavor.
Additionally, I would like to express my gratitude to the faculty and staff of Information Systems for their invaluable assistance, educational materials, and supportive atmosphere, all of which have significantly enhanced my educational journey. Their contributions were crucial to the successful completion of this study. In conclusion, I express my gratitude towards my fellows for their support, intellectual stimulation, and positive reinforcement, all of which have contributed to the overall enrichment of this scholarly expedition. UNIVERSITY OF INFORMATION TECHNOLOGY Advanced ADVANCED PROGRAM echaciatinr IN INFORMATION SYSTEMS Program THESIS PROPOSAL THESIS TITLE: Detecting sources using YouTube original videos to manipulate contents with other purposes Advisor: Professor Do Phuc Email: phucd @uit.vn Master Nguyen Thi Kim Phung Email: phungntk @uit.vn Duration: From 01/09/2023 to 30/12/2023 Students: Dang Truc Lam - 19521736 Phone number: 0334610254 Email: 19521736 @gm.vn Contents: : Misinformation and manipulated content have become more widely distributed as a result of the exponential expansion of digital media[1][2], particularly on platforms such as YouTube[3][4][5].
The objective of this study is to devise a technique for indicating the originators of misinformation and manipulation videos on YouTube. The purpose of this research is to develop a framework for the identification of manipulated videos through the application of deep learning algorithms and the analysis of their characteristics. Enhancing xi the efficacy of approaches to counter misinformation on digital platforms will be facilitated by the results of this study. The first part of the study looks at how common misinformation (manipulated/ modified information) is on digital platforms, focusing on YouTube as a major way for changed content to get around.
After carefully reading all the previous research on the topic, this study finds problems with the current methods of detection and suggests a new approach that uses deep learning algorithms to make detection more accurate. The method involves collecting a diverse set of YouTube videos that include both real and fake content. Videos are broken down into parts like image frames, and audio patterns so that deep learning models can be implemented. Objectives: Analyzing the distinctions between original YouTube videos and modified videos.
This works as a fundamental basis for information security and data integrity on digital platforms. Scope: The subject matter is implemented in order to analyze YouTube videos, video similarity comparison. Methodologies: Applying algorithms and deep learning models specifically designed for Natural Language Processing (NLP) to address the challenges of YouTube2Text, etc. Assessing the vector parameters to ascertain if the input video has been modified.
Simultaneously, evaluating the performance of existing models to determine the most optimal model for the procedure. Expected result: Proposal for an optimized method for detecting videos that spread misleading/ modiefied information with the intention of manipulating viewers. Shu, “Combating Health Misinformation in Social Media: Characterization, Detection, Intervention, and Open Issues. Available: http://arxiv.
Shah, “False Information on Web and Social Media: A Survey. Available: http://arxiv. Kirchhof, “YouTube as a source of misinformation on COVID-19 vaccination: a systematic analysis.,” BMJ Glob Health, vol. Tang et al., ““Down the Rabbit Hole’ of Vaccine Misinformation on YouTube: Network Exposure Study.,” J Med Internet Res, vol.
Srba et al., “Auditing YouTube’s Recommendation Algorithm for Misinformation Filter Bubbles,” ACM Trans. Research plan: - Identify and collect a diverse dataset of YouTube videos. - Preprocess the data, including cleaning and organizing the videos for analysis. - Extract features from the collected videos, including images and audios.
- Develop deep learning models for Youtube2Text, Doc2Vec, and document comparisons. - Evaluate the performance of the trained models. - Compare model results with other related works. xiii Approved by the advisor(s) Ho Chi Minh city, 01/09/2023 Signature(s) of advisor(s) Signature(s) of student(s) Prof.
Do Phuc Dang Truc Lam MSC. Nguyen Thi Kim Phung XIV TABLE OF CONTENTS ACKNOWLEDGMEN TỒ.006096 xi TABLE OF CONTIEN TỀ. 000001000600090804800006 XV LIST OF EIGU RES. 00000 60068008900 xvii LIST OF TA B.
I0 0000089089081 080 XX LIST OF ABBREVIA TIONN. ng 000008966 xxi ABSTRRACTT,. cóc sọ cọ HH 000910000 00000. Problem statement and chaÏlenge€s.
Problem sfaf€Imeii(.- -- -- «c1 s11 9 119 111v HH HH ệt 3 I0). Surveying methods for analyzing video COnIf€T. Study objectives and SCOpC.cceecesseeseeseseeeesceeseesseceseeeaesecesseeseeeaeesaeeeaeeees 7 In ODjeCtIVES. 8 Chapter 2 — Theoretical Background.
Current approaches to solve the problem. Approaches to video content based on Âudo. Approaches to video content based on Image. Vector comparison methodology .- s6 +skssserseeereevee 39 Chapter 3 — Experiments and Evaluations.
Lack of video dataset SUTVCY. HH HH HH nhh 44 3. Alternative datasets for the problem. Implementation process eSCTID{IOH.
Audio arnaÌYS1S. so c1 HH HH Hệ 49 3. Video frame aïiaÏS1S.- - -- c2 13923133 1119111 911 9v ng re 51 Chapter 4 — Conclusion and Future đỈT€CẨÏOIIS. Achievements and ÏIm1tatÏOTS.
-- c2 1121111211110 1 11011110111 011 1 11 1v HH HH Hy 56 4. G5 <5 HH HC 0000000000000 80 58 APPENDICES. 0c cụ Họ Họ.040004 0 65 Xvi LIST OF FIGURES Figure 1. Topic-wide distribution of videos [7].
Pipeline of Jagtap et al. Multi-class classification: Best models and embeddings with highest weighted Fl-score for each topic. --- 6 6 St 1 9112321 91 5119111 1v HT HH gà nưệp 5 Figure 1. Summary datasets using 1n [ LÚ],.-- - - - 2511911911931 91 11 1 9v ng ng 5 Figure 1.
Evaluation of fine-tuning base transformers model &. Baevski et al.- -- --+--+++s++ex+eexsexsersessserses 9 Figure 2. Conformer encoder model architecture [19]. Overview of Whisper methodology architecture [2 Í].
Word Error Rate (WER) distribution of Whisper and state-of-the-art commercial and open-source ASR system in long-from transcription, using 7 distinct long form English-only afaS€fS.- (c1 1191 nh HH Hàn 14 Figure 2. A timeline overview of the latest advancements in fundamental network structures and models for word embeddings, starting from 2013. Bag of words model [2S] .-- --- --- 2 + + E1 E939 E91 93 31 11 v1 gi ng 16 Figure 2. Example of drawback of BOW [26 ].
- c5 + kg re, 16 Figure 2.- --- --- + + + < + E111 911911 91 91 91 1 1111 nh HH nh 17 Figure 2. The difference between CBOW and Skip-gram[29].- ¿s55 «<< cs++ 18 II0410290808xÀ/219)/0ii13. PV-DBOW model. - -- 6 5 6 21 2119 91191 911911 vn nh nu ng gh nnrnry 19 Figure 2.
Performance comparison on SST đafaS€(. Performance comparison on IMDB dataset. Example of 3 paragraphs .- --- <1 131911919 111910 1911 vn HH ngư 23 Figure 2. Results of example in Figure 2.
Video to frames using OpenCV example [39] .