VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY FACULTY OF COMPUTER SCIENCE AND ENGINEERING GRADUATION THESIS AN INTELLIGENT TRAFFIC SYSTEM BASED ON DATA ANALYSIS OF TRAFFIC DENSITY Major: COMPUTER ENGINEERING THESIS COMMITTEE: Computer Engineering 1 SUPERVISOR: Dr. Pham Hoang Anh CO-SUPERVISOR: Assist. Kieu Vu Thanh Tung REVIEWER: Assoc. Tran Minh Quang Student 1 Ho Thanh Binh 2010929 Student 2 Le Nhat Dang 2052950 Student 3 Tran Khoa 2052541 Ho Chi Minh City, May 2024 ĐẠI HỌC QUỐC GIA TP.HCM CỘNG HÒA XÃ HỘI CHỦ NGHĨA VIỆT NAM TRƯỜNG ĐẠI HỌC BÁCH KHOA Độc lập - Tự do - Hạnh phúc KHOA: KH & KT Máy tính NHIỆM VỤ LUẬN VĂN/ĐỒ ÁN TỐT NGHIỆP BỘ MÔN: KTMT Chú ý: Sinh viên phải dán tờ này vào trang nhất của bản thuyết trình HỌ VÀ TÊN: HỒ THANH BÌNH MSSV: 2010929 HỌ VÀ TÊN: LÊ NHẬT ĐĂNG MSSV: 2052950 HỌ VÀ TÊN: TRÂN KHOA MSSV: 2052541 NGÀNH: KỸ THUẬT MÁY TÍNH LỚP: 1.
Đầu đề luận văn/ đồ án tốt nghiệp: An Intelligent Traffic System based on Data Analysis of Traffic Density 2. Nhiệm vụ (yêu cầu về nội dung và số liệu ban đầu): - Khảo sát và nghiên cứu các kiến thức liên quan đến hệ thống ITS - Khảo sát, nghiên cứu mô hình học máy để tìm đa cực trong chuỗi thời gian để áp dụng trong phân tích dữ liệu giao thông - Thực hiện một số thử nghiệm để đánh giá tính khả thi chức năng dự đoán lưu lượng giao thông dựa trên phương pháp phân tích đa cực - Điều chỉnh và thực hiện mô phỏng để đánh giá giải pháp đề xuất 3. Ngày giao nhiệm vụ: 08/01/2024 4. Ngày hoàn thành nhiệm vụ: 20/05/2024 5.
Họ tên giảng viên hướng dẫn: Phần hướng dẫn: 1) Phạm Hoàng Anh 100% 2) Kiều Vũ Thanh Tùng 100% Nội dung và yêu cầu LVTN/ ĐATN đã được thông qua Bộ môn. CHỦ NHIỆM BỘ MÔN GIẢNG VIÊN HƯỚNG DẪN CHÍNH (Ký và ghi rõ họ tên) (Ký và ghi rõ họ tên) PHẠM QUỐC CƯỜNG PHẠM HOÀNG ANH PHẦN DÀNH CHO KHOA, BỘ MÔN: Người duyệt (chấm sơ bộ): Đơn vị: Ngày bảo vệ: Điểm tổng kết: Nơi lưu trữ LVTN/ĐATN: Commitment We confirm that this specialized project represents a rigorous and honest scientific endeavor undertaken by our team under the guidance of Ph.D Pham Hoang Anh and Ph. Every aspect of the project, excluding explicitly cited and referenced sections in the bibliography, is a product of our own efforts. We assume full responsibility for any potential misconduct related to the content of our Specialized Project.
Acknowledgement This specialized project attains its culmination under the scholarly guidance of Ph.D Pham Hoang Anh, the Faculty of Computer Science and Engineering at Ho Chi Minh City University of Technology, Vietnam National University. Our profound appreciation extends to Ph.D Pham Hoang Anh for his erudition, strategic discussions, invaluable recommendations, and the creation of an enabling environment that facilitated the successful fruition of our project. We also want to show our gratefulness to Ph.D Tung Kieu, also our project supervisor, for the critical advice on proposed methodology and data result analysis, as well as knowledge discussion on deep learning and time series. Without their wholehearted supports and valuable bits of advice, we could not have accomplished this project.
Throughout the intricate trajectory of our research, substantive infrastructure support was graciously provided by online resources and studies. Additionally, elucidations from our aca- demic predecessors contributed significantly. We express sincere gratitude for the wholehearted collaboration and support received from our mentors and esteemed colleagues, a pivotal factor in the accomplishment of this project. Special acknowledgment is reserved for our esteemed evaluators, whose discerning feedback and scholarly insights enriched the refinement of our specialized project.
Our sincere gratitude is extended to the distinguished faculty members of the Faculty of Computer Science and Engineering at Ho Chi Minh City University of Technology - Vietnam National University. Beyond the dissemination of specialized knowledge, they have fostered an environment conducive to our intellectual and professional development during this transforma- tive phase. We extend our best wishes to our mentors for continued good health, professional success, and their enduring commitment to shaping the future through academic guidance. With earnest appreciation, we express our gratitude.
Ho Thanh Binh Le Nhat Dang Tran Khoa Abstract Through human development, the traffic system has evolved drastically, creating a well- constructed but underwhelming complex infrastructure. Requiring highly efficient traffic man- agement system, thus, Witness of modern practices ITS short for Intelligent Traffic System foundation. An ITS is expected to innovate services relating to different modes of transport by promoting better traffic coordinating with decision-making aid suggestions; sensing abnor- malities occurrences. These tasks orbit around peeling data information to provide data-based insights as guidelines for the system, which even with the aid of modern data analyzing tools still find challenging.
To contribute to the two mentioned tasks, we presented a data analyzing method using traffic flow time series to capture the correlation involving multiple traffic nodes data, called multipole. Since each traffic node often has a close circle involving several others sharing strong relationships, multipole aid searching for these groups with evaluation on their strength and contribution of each participant. Thus, provides meaningful insights backed by various node data, guiding strategy for efficient traffic management and warning for abnormality occurrences. State-of-the-art algorithms searching for multipole although well-instructed and efficient still have limited performance in large-scale datasets due to their nature of combination solving methods.
To all those aforementioned ends, we proposed a neural network learner that works as a top-down strength multipole searching approach as a contribution to drawback presented in popular algorithms. The model learns to find combinations for multipoles of certain size range regarding input signal, qualifying the definition of multipole. Providing an efficient option with ideal running time, and training time does not scale proportionally with target output size increment. We conduct experiments on two standard benchmark datasets, PEMs08 and NYC-TOD.
The experiment results show that our proposed methods has edge in multipole strength priority searching and efficiency compared to other methods published in top-tier journals. Furthermore, through the analyzing of experiments, we show an interesting observation holds ground for the practicality of multipole in traffic management and abnormality sensing. In sum, we conclude that multipole found a proper stand as a data analyzing method for the mentioned traffic tasks, and our proposed model satisfies the need for multipole efficient searching approach.1 Topic Selection Rationale .2 Exiting Work Limitation .3 Purpose Of Project .1 Traffic management and abnormally detecting based on insights from multipole analysis .2 Multipoles searching deep learning based method .3 Normalized Linear Combination .4 Least Variant Normalized Linear Combination .7 Multipoles and Maximal Multipoles .8 Pattern Similarity in Multipole .10 Brute Force Algorithm .1 Deep learning model .2 Unsupervised De-Mixing and Weakly Supervised De-Mixing .1 Unsupervised De-Mixing .2 Weakly Supervised De-Mixing .4 Principle Component Analysis (PCA) .1 What is Principle Component Analysis .3 Covariance Matrix Computation .4 Identify Principal Components through Eigenvalues and Eigenvectors .5 Feature vector creation .6 Recast the Data Along the Principal Components Axes .5 Eigenanalysis-based Approaches .6 Error-in-variables .2 Deep Multipole De-mixing .3 Data Insight Evaluation .1 Summary of our contributions. 54 List of Figures 1.1 The process of collecting raw traffic density data and convert it into meaningful data of an Intelligent Transportation system .2 Insight from Data Analyzing where 3 poles A, B, C have strong correlation and similarity in patterns .1 An example of airline passenger data over a period of time.2 An example of traffic data.1 example of a multi-layer perceptron.2 Plotting Sigmoid function and grad of Sigmoid function .3 Plotting tanh function and grad of tanh function .4 Plotting ReLU function and grad of ReLU function .5 Error on training and testing data, taken from Deep Residual Learning for Image Recognition, 2016 .6 Example of residual block .7 Example of ResNet .8 Conditional GAN architecture .9 Example of a Conditional Generator and a Conditional Discriminator in a Con- ditional Generative Adversarial Network.1 Workflow of CoMEt .2 Model receive input then embedded into latent spaces content features need for de-mixing into possible component time-series.
Decoder inputs multiplied with coordinated probabilities predicted by encoder to reconstruct input.3 Workflow of proposed approach .1 3-poles found by CoMEt consisted of flow time series of node 18-th, 21-st and 52-nd in NYC-TOD dataset.2 Node 21-st time series and its reconstruction formed by 18-th and 52-nd combi- nation, sharing high similarity in pattern.3 3-poles found by DMD consisted of flow time series of node 48-th, 12-st and 38-th in NYC-TOD dataset. 51 University of Technology , Ho Chi Minh City Faculty of Computer Science and Engineering 5.4 Node 48-th time series and its reconstruction formed by 12-st and 38-th combi- nation, sharing high similarity in pattern. 52 GRADUATION THESIS - SEMESTER 232 - ACADEMIC YEAR 2023 - 2024 Page 0/56 University of Technology , Ho Chi Minh City Faculty of Computer Science and Engineering Chapter 1 Introduction Intelligent Transportation Systems (ITS) refers to the integration of advanced information and communication technologies into transportation infrastructure and vehicles. The primary goal of ITS is to enhance the efficiency, safety, and sustainability of transportation networks.
Representing the forefront of modern practices in advancing traffic management, leveraging cutting-edge technologies and data-driven approaches to optimize various modes of transport and enhance urban mobility. The concept of our ITS transcends traditional methods by harnessing the power of real-time data analytics, predictive algorithms, and decision-making aids to orchestrate traffic flow efficiently and address emerging challenges in urban mobility. In this project, we applied traffic density data analysis to various applications in Intelligent Transportation Systems (ITS). We use traffic density data to delve deep into the intricacies of dynamic traffic management and anomaly detection, with a focus on analyzing the complex web of correlations among diverse data sources.
By leveraging insights learned from intercon- nected traffic nodes and scrutinizing time-series data on traffic patterns, we aim to develop a comprehensive understanding of the underlying dynamics driving urban mobility. Central to our approach is the utilization of advanced data analytics techniques to extract insights (traffic patterns, anomalies) from vast and diverse datasets. Through the integration of machine learning algorithms, statistical models, and data visualization tools, we endeavor to uncover hidden patterns, detect anomalies, and generate informed recommendations for traffic coordination and management. Through our interdisciplinary approach and collaborative efforts, we aspire to pave the way for a more efficient, sustainable, and resilient transportation infrastructure.
By harnessing the power of intelligent transportation systems and data-driven decision-making, we believe that we can unlock new opportunities for enhancing urban mobility, improving quality of life, and shaping the future of transportation in cities around the world. GRADUATION THESIS - SEMESTER 232 - ACADEMIC YEAR 2023 - 2024 Page 1/56 University of Technology , Ho Chi Minh City Faculty of Computer Science and Engineering 1.1 Topic Selection Rationale The traffic system has radically changed as a result of human growth, and the complex infrastructure that has resulted in calls for extremely effective traffic control systems. Despite advancements, traditional methods are insufficient for managing modern transportation chal- lenges, prompting the need for innovative solutions like Intelligent Traffic Systems (ITS). An ITS aims to enhance traffic coordination, decision-making, and anomaly detection by leveraging data-driven insights from various modes of transport.
However, these tasks remain challenging due to the complexities involved in analyzing vast amounts of traffic density data. In order to analyze large amounts of traffic data in effective way, our group devises a strategy utilizing multipole analysis using traffic flow time series, a novel method for data analysis. With the help of multipole analysis, it is possible to obtain valuable insights for traffic management and anomaly detection, contributing to more efficient and responsive urban mobility systems. Finding multipoles in traffic flow data is not easy work and the existing algorithms for identifying such multipoles face limitations when dealing with large-scale datasets.
