Vietnam National University Ho Chi Minh City University of Technology Faculty of Computer Science & Engineering GRADUATION THESIS PREDICT URBAN TRAFFIC CONDITION AND ESTIMATE TRAVEL TIME MAJOR: COMPUTER SCIENCE Supervisor: Assoc. Trần Minh Quang Reviewer: Assoc. Nguyễn Thanh Bình Student 1: Bùi Minh Kiệt – 1852115 Student 2: Huỳnh Ngọc Nhật Quang – 1852118 HO CHI MINH CITY, 09/2022 DECLARATION OF AUTHENTICITY We declare that this thesis is our own work, conducted under the supervision and guidance of Associate Professor Tran Minh Quang. The idea, implementation, and result of our work are legitimate and have not been published in any form prior to this.
All materials and research mentioned, used, and improved upon within this report are guaranteed to be referenced in the bibliography section with their respective authors/ owners. We take full responsibility for the authenticity of our research. In any case of plagiarism, we stand by our actions and will be responsible for it. Ho Chi Minh City, September 2022 Research group i ACKNOWLEDGEMENT This project cannot be completed without the support and guidance we have received from our lecturers at Ho Chi Minh City University of Technology, partic- ularly from the Faculty of Computer Science and Engineering.
We would like to express our gratitude to the university for the lessons, encour- agement, and opportunities that we have received during the years spent studying here. The knowledge and experience we have had would not only be valuable for our professional careers onward, but would also help us develop and improve ourselves in society. We would like to offer our foremost acknowledgment to Dr. Tran Minh Quang, for having wholeheartedly trusted and guided us to complete this thesis and for providing us the valuable resource and advice to improve our work.
We also would like to thank our families and friends for the support throughout the years, which have helped us both mentally and emotionally - these are all crucial for one to be motivated to achieve the best things in life. Finally, we would like to wish our instructor and all the lecturers at Ho Chi Minh City University of Technology the best in the future. Ho Chi Minh City, September 2022 Research group ii ABSTRACT The project focuses on "Predicting urban traffic condition and estimate travel time", which revolves around developing and improving the current BKTraffic system. The main objectives are to research and implement methods to estimate traffic status and travel time in Ho Chi Minh City, as well as to aid residents by improving the performance and experience of the BKTraffic application.
The main contents of this thesis are as followed: • Chapter 1: Introduction, overview project information, and detail problem statements and objectives. • Chapter 2: Review and outline of related research, projects, and implemen- tations. • Chapter 3: Description and analysis of current BKTraffic system, including system architecture, database structure, server, and APIs structures, as well as web & mobile application overview. We also briefly discuss some evalua- tions on our server and the benchmarking methods that we utilize.
• Chapter 4: Theoretical background, solution design, and implementation of predicting traffic status and travel time of each segment using available data. • Chapter 5: Theoretical background, solution design, and implementation of improving and optimizing database performance with sharding. • Chapter 6: Design, implementation, and evaluation of current server and application updates to improve user experience. • Chapter 7: Conclusion of completed works, analyze results and improve- ments, as well as detailing potential improvements on the project.
iii Contents List of Tables 2 List of Figures 4 1 Introduction 5 1.2 Initial perspectives and approach .3 Scope and ojectives. 8 2 Related Works 9 3 System Overview 13 3.3 Server and APIs structure. 23 4 Traffic status prediction 25 4.1 Current system overview and problems .1 System traffic status display method .2 Current approach on traffic status prediction .2 Proposed approach and theoretical background .1 Regression training model for the ETA problem .2 Artificial Neural Network for regression .3 Hyperparameter tuning with Bayesian Optimization .2 Feature Engineering and Data collection .3 Data mining and model for speed prediction .1 Overview of the traffic prediction analysis server .2 Data collection and preprocessing .3 Neural network construction and parameter tuning .5 Results and Evaluation .1 Data collection and feature engineering .2 Model testing method .3 Model performance benchmark .4 System load testing performance evaluation .5 Performance improvement analysis .6 Limitation and Improvement plan. 49 5 Database scaling and optimization 50 5.1 Overview and problem statement .2 Theory: MongoDB Sharding .1 Sharded cluster deployment for BKTraffic database .5 Result and Evaluation .1 Live weather report .2 BKTraffic-Analyzer Server API Structure .3 MongoDB Sharding on BKTraffic server deployment .3 Future development and plan.
88 References 90 1 List of Tables 3.1 Level of Service explanation .2 BKTraffic database capacity overview .3 BKTraffic API result status .1 Old model’s input form before encoding and normalization .2 Tuned hyperparameter choice .3 BKTraffic-Analyzer model predicted speed MAPE benchmark .4 BKTraffic-Analyzer model LOS accuracy benchmark .5 BKTraffic load testing benchmark .1 Current API response time for some selected traffic status query on BKTraffic .2 Comparison on number of shards on Segments collection by average elapsed time .3 Detailed comparison between different sharding scenarios for _id query on Segments collection .4 Detailed comparison between different sharding scenarios for start_node query on Segments collection .5 Sharded database load testing benchmark .1 API Structure of BKTraffic-Analyzer speed estimator .2 Weather options for traffic status report form .3 API definition for model inference. 83 2 List of Figures 1.1 Traffic jam in the morning in Ho Chi Minh City .1 Camera image data from HCMC Department of Transportation .2 General maps of HCMC Department of Transportation’s Web Portal 11 2.3 BKTraffic Website User Interface .4 BKTraffic Mobile App User Interface .2 ERD view of main BKTraffic components .3 Example of ReactJs life cycle .1 BKTraffic Form report functionality .2 BKTraffic Voice report functionality .3 VOH Station live traffic report display .4 Prediction model deployment from previous research group .5 Simple Perception layer .6 Multilayer Perceptron model .7 Bayesian optimization demonstration .8 BKTrafficAnalyzer model architecture .9 TomTom’s Traffic flow user view in map .10 Overall architecture of BKTrafficAnalyzer crawler service .11 Traffic prediction data mining model deployment .12 New model’s input form before encoding and normalization .13 Lowest zoom level view of public data service .1 Illustration of the most simple advantage of sharding: increased ca- pacity .2 Interaction of components within a sharded cluster .5 Zones in sharded clusters example .6 MongoDB Sharding Deployment .7 Overview of dumped database, which are conveniently stored as BSON and JSON file .8 Illustration of MongoDB’s load balancer work mechanism by dis- tributing chunks to different shards .9 Sharding information of the initial 2-shard deployment .10 JMeter testing result sample .11 Normalized response time distribution graph for test case 2 .12 Normalized response time distribution graph for test case 3 .1 Existing BKTraffic form report interface .2 OpenWeatherAPI detailed information for HCMC .3 Sequence diagram of weather display use case .4 Schematic design of WeatherInfo collection .5 User Interface view of weather display on BKTraffic web app .6 Sequence diagram of traffic status form submission use case .7 New ERD diagram of major database collections after SegmentRe- ports update .8 New traffic report form with weather options listed .9 New traffic report form general view .1 Project overview The world is developing in an incredibly rapid way, both in quality and quantity of life. The fourth industrial revolution (4.0) has brought forth significant changes to all aspects of life, and it cannot be denied that the world in the past few decades has shifted and developed in various different directions [1]. One of the most dis- cussed topics during this era of industrial revolution includes traffic, especially in the context of developing countries with rapidly increasing populations and living standards.
In Vietnam particularly, there have been multiple improvements and innova- tion efforts made to improve the quality of urban traffic around the cities - no- tably including improving urban infrastructure, encouraging public transport us- age, restructuring urban population planning, and incorporating technology ad- vancement [2]. Despite these development plans, currently traffic movement still contains some of the most annoying problems among the urban population. Some of the most sig- nificant over the years are: traffic congestion (see figure 1.1), unbalanced traffic flow, unsafe road infrastructure, and traffic accidents. Traffic congestion, in partic- ular, has been causing a lot of damage and regression to the overall development of our country’s urban life in general.
There have occurred a lot of studies and journals on the cause and effects of urban traffic jams in Vietnam. Most of them attribute the current state of traffic to the exploding population, especially in major cities like Ho Chi Minh City and Hanoi; as well as imperfect urban planning for transport around the city. It has also been pointed out that the damage these traffic jams result in is also proportional to the cause, which is the population density. It makes average travel time around cities increase, aggravates the use of energy and the state of air & noise pollution.
These not only affect the overall quality of life experience among the residents, but also seriously threatens the consistent and stable development of the country in the far future. From an estimation of around four years ago, each year HCM City lost around 1.2 million of public service hours, 1.3 billion USD from traffic congestion and 2.3 billion USD from environmental pollution [3]. According to HCM City Department of Transport, the land allocated for traffic in the city centre is lower than expected 5 Figure 1.1: Traffic jam in the morning in Ho Chi Minh City.vn] (only a third of the standard), which leads to an average of 2.4 billion VND lost for every hour of traffic jams. Despite the restricted travel policy during the COVID-19 pandemic, traffic prob- lems still persist and require attention from the authority.
Overall, resolving traffic congestion is still a hot active issue in Vietnam and countries around the world, and solutions or supports that can help alleviate the problems are still one of the most prioritized works at the moment. And it is also the inspiration for us to research and develop the project in our thesis proposal.2 Initial perspectives and approach Firstly, as we have mentioned before, there have been four main directions to improve urban traffic situations in Vietnam. We will review and explain the insights of each direction. • Improving urban infrastructure, by improving roads and intersections quality and increasing traffic flow capacity.
This is indeed one of the most research projects in civil engineering, but it is also not a reliable directive since the allocation of lands and space for traffic is limited, and the supply improvement is not subjected to be able to allocate the exploding increase of the population demand in the future. • Encouraging public transport usage. This includes investing in developing and improving the current state of public transport infrastructure, and implement- ing policies to increase the use of public transports like buses and subways. However, this solution requires substantial planning and investment before 6 realizing the result, and while this would be effective and environmentally helpful, the scope of its time-wise and resource-wise is certainly not suitable for our project.
• Restructuring urban planning plan, including redistributing the population density into the suburban areas of the city and improving quality of life in these areas. This is certainly one of the most prioritized direction, because it will benefit a lot of aspects beyond just the traffic problems. However, the scale of the project in this direction is also the largest, which can only be taken step-by-step and by authorities with definitive plans. • Incorporating technology in improving traffic situations of residents.