VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY COMPUTER ENGINEERING DEPARTMENT PHAN TRAN QUOC DAT VÕ QUOC HUY GRADUATION THESIS RESEARCH AND IMPLEMENTATION OF DETECTING AND TRACKING SYSTEM OF VEHICLE ON NATIONAL WAYS ENGINEER OF COMPUTER ENGINEERING HO CHi MINH CITY, 2021 VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY COMPUTER ENGINEERING DEPARTMENT PHAN TRAN QUOC DAT- 16520203 VO QUOC HUY- 16520536 GRADUATION THESIS RESEARCH AND IMPLEMENTATION OF DETECTING AND TRACKING SYSTEM OF VEHICLE ON NATIONAL WAYS NGHIEN CUU VA THUC HIEN HE THONG PHAT HIEN, THEO DOI TOC DO XE TREN DUONG QUOC LO ENGINEER OF COMPUTER ENGINEERING INSTRUCTOR PhD. LAM DUC KHAI HO CHi MINH CITY, 2021 PROTECTION COUNCIL OF THE GRADUATE THESIS Protection council of the graduate thesis, established under decision no 70/QD-DQCNTT dated January 27", 2021 of the Rector of University of Information Technology ACKNOWLEDGEMENTS We would like to give our gratitude to Ph. Lam Duc Khai and Ph. Nguyen Minh Son for the passionate answers whenever we face complicated questions, the useful advice whenever we run into troublesome tasks, the directions whenever we lost track of situations.
We also wish to give our appreciation to all the instructors of UIT computer engineering department for every single lesson which provide us with the knowledge, nurture us to become better people for the society. Finally, we would like to give out big thanks for all the people that constantly assist us in the process of carrying out the research, give us the encouragements in the hard time, and support us both in finance and mentality. This research can not be done without your enthusiastic help. TABLE OF CONTENTS Chapter 1.
CURRENT PROBLEM AND POTENTIAL SOLUTION. ¿St 111 2 SE 1217101 1 111 12 HH re 1 1. Problem and direction Chapter 3. Review of object detection model: 3.
Detail of Yolo Model for object detection 3. The reason Choosing _YoÏO. How this WOTK. 5 cà 221 1212121211112 HH HH 19 3.
Dataset and Training.--- St tt 1010121 H000 hàn 20 3. Dataset and why choOSingE. ¿tt St 20 Kao. Processing raw iIAgCS.
ánh HT HH ngư 2 ky. Evaluate the training r€SuÏ.- -¿--- «+ + St St re 25 3. Review of object tracking algorithms. c1 1 E* ST 1212 H1 HH re.
Processing flow of the SORT. Speed measuring Chapter 4. PROJECT IMPLE TION AND RESULT EVALUATION. Evaluating model base on đetecting distance 39 4.
Evaluating model base on detecting aCCUTaCÿ. Evaluating model base on tracking vehicle. CONCLUSION AND FUTURE WORKK. Future work 51 REFERENCES.
52 FIGURE MENU Figure 1-1: Traffic jam in VietfIam. -- ¿+ + 1 Figure 1-2: Using internet of thing in public transportation %œœ6 Figure 1-3: Idea design Figure 2-1: Intelligent transportation system: Figure 3-1: Image processing in R-CNN Figure 3-2: Image processing in Fast R-CNN Figure 3-3: Left: Region proposal network (RPN). Right: Samples detection of lý00. 10 Figure 3-5: SSD mechanism in training and detecting Figure 3-6: YOLO workflow.
Figure 3-7: YoLo performance Figure 3-8: Darknet framework loads 106 layers for every commands. Figure 3-9: Darknet-53 Figure 3-10: Image by Ayoosh Kathuria Figure 3-11: Image by Valentyn Sichkar(a) Figure 3-12: Image by Valentyn Sichkar(b) Figure 3-13: Total bounding boxes of three difference scales Figure 3-14: Calculate bounding box by using the anchor Figure 3-15: Equation for objecness score - Image by Valentyn Sichkai Figure 3-16: Image of objects detected on HoChiMinh cit Figure 3-17: The result displayed on terminal Figure 3-18: Types ofvehicle used to train model Figure 3-19: Instances label. Figure 3-20: Weather types. Figure 3-21: Annotation format SOCDRHKBVAIUWY Figure 3-22: Interface of labeling Figure 3-23: Services from Collab.
Figure 3-24: Training procedure. Figure 3-25: Chart evaluate after training Figure 3-26: Detail result after training. Figure 3-27: Meanshift illustration Figure 3-28: Steps in the operation Figure 3-29: Content of kalman. Figure 3-30: Processing flow of the SORT.
Figure 3-31: Estimate the speed of vehicles Figure 4-1: Jetson Nano kit Figure 4-2: OpenCv. Figure 4-3: Distance at sunny Figure 4-4: Distance at cloudy. Figure 4-5: Distance at near night Figure 4-6: Distance at rain. Figure 4-7: Distance at rainy nigh Figure 4-8: Video result.
Figure 4-9: Detect object partially obscured. Figure 4-10: Frame 108 case | Figure 4-11: Frame 262 case 1. Figure 4-12: Frame 15 case 2. Figure 4-13: Frame 89 case 2.
Figure 4-14: Frame 133 case Figure 4-15: Frame 256 case Figure 4-16: Id changing TABLE MENU Table 1: Testing results of many models in similar conditions. ---‹ Table 2: Number of imported automobiles on November 2020 Table 3: Specification of hardware on Collab Table 4: Compare the Tracking methods Table 5: Performance of models on different hardwar: Table 6: Jetson Nano kit detai Table 7: Camera detail. Table 8: Distance at sunny. Table 9: Distance at cloudy Table 10: Distance at near night Table 11: Distance at rain Table 12: Distance at rainy night.
Table 13: Video Specifications Table 14: Number of vehicles in practice Table 15: Number of vehicles compare. Table 16: Compare the results ABSTRACT We made this research with the hope to provide a device that can help solve in improve the quality of people when traveling on roads in Vietnam. Thanks to the development of deep learning and computer version, main functions ofthe device are detection and tracking popular vehicles on national roads, in addition, estimate the distance and speed as the same time. The system will have the following direction: - The system will detect various kinds of vehicles using Yolo version 3.
Input can be available videos or transmit through camera attached to the hardware. - The detected objects will be tracking and estimating speed by applying algorithms. According to the expected result, performance of the detecting process can reach about 80% and the ability to calculate speed is added. CURRENT PROBLEM AND POTENTIAL SOLUTION 1.
Problem statement Our country is at the stage of developing in many fields, some of noticeable ones are about human, science, technology,. Through the process, there is no other solution but to be headstrong and deal with challenges born along the way. Figure 1-1: Traffic jam in Vietnam A large-scale problem we could mention of is the national transportation. So far, with 128 national roads having the total length of 17.530 kilometers and a tremendous number of vehicles that are increasing day by day, one might easily think about the hardship for the Ministry of Transport to manage such complex traffic.
Fortunately, the improvement of information technology has taken effect. With the application of deep learning, particularly detection and tracking methods, we can give assists to controlling traffic more adequately. One great detecting and tracking system can not only support the management of traveling but also raise the citizens’ consciousness of safety when participating in traffic flow. The idea From primitive forms like walking or riding bicycle on trails, vehicles with four wheels appear regularly on the asphalt roads which are prolonged from time to time.
Beside positive aspects such as the improving of everyday life quality, people have to confront trouble incurred from that growth. Many types of vehicles meaning more tule need to establish to keep all of them under control, the frustration of people when they get stuck in a traffic jam which can last for a few hours after a day of tiring work, or the lack of sense of responsibility and safety for themselves and the others, all of Figure 1-3: Idea design With the application of deep learning to detect and track vehicles, the amount of work will be able to reduce. Having a system to analyze the number of vehicles in specific frame of time so we can give citizens announcements about the situation on the road so they could avoid driving on jamming roads. For the traffic police, having an accurate information will put them in the right destination to give out traffic commands, or in the office, they can detect vehicles that break the rules.
Not only solving external issues, but the project may also join in fixing internal problems. People knowing about the system will have to behave themselves if they do not want to have punishments, gradually, they will form a civilized habit, care more for their own safety and the others’ as the same time. Methodology About this project, we propose three main steps that need to acquire: — The first one is to understand the basic concept of Deep learning. Foundation is always an important element when approach new knowledge.
Our group had gathered necessary information from various reliable sources: learning websites on the internet, articles of many previous researches, experience from qualified people,. — The second one is the implementation ofthe project. This is the main stage of the project as it carries multiple tasks to be accomplished. For the hardware will have a crucial role in deciding the performance of the system, a choice need to be make considered about the functionality and price.
After making an overall view at the marketplaces, NVidia Jetson Nano kit seems to be a reasonable pick because of the efficiency in both price and performance. After setting up the hardware with every requirement, YOLO algorithm, a state of art Object Detector, had been installed and run demo to see the result. Next step is collecting the dataset for our purpose. Dataset is considered one of the most basic factors to evaluate if the model can be applied to reality or not.
The more resourceful dataset with a precise annotation will give the model an ability to learn deeply about the features of objects that we desired to detect, in order word, the result will gain better accuracy for this project. The intention of this dataset is from Vietnam, for Vietnam as most of training data will be collect in our nation so model can get used to environment in this country. — Finally, the last target is training and testing the model. Theory is usually different from reality, so testing will never be an ignorable subject.
After all, this project is mean to be able to be applied in real world. After testing functionality and packing vital components, we bring the system to suitable environment for the validation. Domestic The human eye is structured in the following parts: — ITS (intelligent transportation system) for Vietnam Expressway has been publicly available in March 2017. This project is the fruit of cooperation of a consortium of Japanese companies led by Toshiba.
The man in charged claimed the system including cutting-edge information processing technology for analyze vehicles on the road, which results in reducing the disruption along with the network and inconvenience for its users. ~ ——Ÿ_ ` Figure 2-1: Intelligent transportation system — Da Nang smart camera system has been carrying out as a foundation to create a smart city in the future. The system can run 24/24 hours, detect two kinds of vehicles (car and motorbike), and can report traffic violations. The scale of the project is at national level: there are nearly 50 locations that are installed smart camera, the total number of traffic cameras in the city is 143 with 125 surveillance cameras, 9 speed testing cameras and 9 observation cameras.
Foreign A Cascade of Boosted Generative and Discriminative Classifiers for Vehicle Detection conducted by Pablo Negri, Xavier Clady, Shehzad Muhammad Hanif, and Lionel Prevost showed a cascade of boosted classifiers for vehicle detection in scene image on the road. The project studied two main features: Haar-like features and HoG (histogram of oriented gradients) features, where Haar-like features are used to construct discriminative weak classifiers while the other ones are used to construct generative weak classifiers. The fusion detector combines the advantages of both Haar and HoG detectors and achieves a high correct detection rate of 94% and a small number of false alarms rate per image of 0. The result had been evaluated on 2.2 GHz processor and had not been tested in practice.