MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING GRADUATION THESIS AUTOMATION AND CONTROL ENGINEERING TECHNOLOGY RESEARCH, DESIGN AND CONSTRUCT AN AUTONOMOUS GOLF-CART BASED ON MULTI-TASK MODEL AND GPS NAVIGATION ADVISOR : ASSOC. LE MY HA STUDENTS: TA TRAN NHAT MINH SKL011633 Ho Chi Minh City, June 2023 MINISTRY OF EDUCATION AND TRAINING HCMC UNIVERSITY OF TECHNOLOGY AND EDUCATION Faculty of Electrical and Electronics Engineering -------------------- FINAL THESIS RESEARCH, DESIGN AND CONSTRUCT AN AUTONOMOUS GOLF-CART BASED ON MULTI- TASK MODEL AND GPS NAVIGATION Major: Automation and Control Engineering Technology Student: Ta Tran Nhat Minh Student ID: 19151034 Acad. Year: 2019 – 2023 Advisor: Assoc. Le My Ha Ho Chi Minh city, June 2023 BIÊN BẢN CHỈNH SỬA ĐATN HK2 NH 2022-2023 NGÀNH CNKT ĐK-TĐH Tên đề tài: Research, design and construct an autonomous golf-cart based on multi-task model and GPS navigation.
Tên SV: Tạ Trần Nhật Minh MSSV: 19151034 Tên GVHD: PGS. Lê Mỹ Hà Nội dung chỉnh sửa STT Yêu cầu chỉnh sửa (GVPB và của Nội dung đã chỉnh sửa (trình bày nhóm đã HĐ) chỉnh sửa gì, tại mục nào, trang bao nhiêu) 1 Chỉnh sửa lại kích thước hình Đã chỉnh sửa kích thước hình 3. 2 Chỉnh sửa lại lưu đồ kết hợp GPS Đã thêm phương pháp đánh lái vào lưu đồ cho và Camera luồng sử dụng GPS. Hình được chỉnh sửa là hình 4.
Xác nhận GVHĐ Tp. HCM, ngày 26 tháng 7 năm 2023 (Chèn chữ kí và ghi rõ họ tên) (Chèn chữ kí và ghi rõ họ tên) i HCMC University of Technology and Education SOCIALIST REPUBLIC OF VIETNAM Faculty of Electrical and Electronics Independence – Freedom– Happiness Engineering ---o0o--- Department of automatic control THESIS ASSIGNMENTS Advisor: Assoc. Le My Ha Student: Ta Tran Nhat Minh Student id: 19151034 Class: 191512A Major: Automation and Control Engineering Technology 1. Project title Research, design and construct an autonomous golf-cart based on multi-task model and GPS navigation.
Project’s content • Research documents, papers, books to find out the direction for solving thesis problems. • Fix the mechanical and electrical problems from my prototype model. • Design and construct the steering control board system with many electronic components. • Calculate steering strategy for control wheeling.
• Collect and label data for training and testing model. • Build a multi-task network with deep learning approach. • Collect and build a campus road map with GPS data. • Synchronize multitask model and GPS navigation with processor board.
• Collect and evaluate the experiment results. • Write the thesis. Assignment date: February 15th, 2023 4. Complete date: June 30th, 2023 CHAIR OF PROGRAM ADVISOR (Sign with full name) (Sign with full name) ii HCMC University of Technology and Education SOCIALIST REPUBLIC OF VIETNAM Faculty of Electrical and Electronics Independence – Freedom– Happiness Engineering ---o0o--- Department of automatic control ADVISOR’S COMMENT SHEET Student: Ta Tran Nhat Minh Student id: 19151034 Class: 191512A Major: Automation and Control Engineering Technology Project: Research, design and construct an autonomous golf-cart based on multi- task model and GPS navigation Advisor Assoc.
Le My Ha COMMENT 1. Thesis’s content: This thesis consists of six chapters and 100 pages. The proposed system is the combination of multiple sensors. The real system is successfully completed the objectives in the proposal.
Advantages: The system has the ability to aid UTE students and lecturers in their mobility within the campus. Successfully build a multitask model with the control technique and mechanism design for guiding self-driving cars. All the sensors used for this project are low-cost devices. The execution time for autonomous mode is suitable for real-time applications.
Disadvantage: The system is unable to function autonomously without the presence of an operator due to the need for upgrades and improvements in various aspects. The accuracy and stability of the system stops at an acceptable level in basic outdoor environments. Propose defending thesis: Approved 5. Overall evaluation: Excellent 6.
Mark: 10 (In words:) Ten Ho Chi Minh, June 2023 ADVISOR (Sign with full name) iii HCMC University of Technology and Education SOCIALIST REPUBLIC OF VIETNAM Faculty of Electrical and Electronics Independence – Freedom– Happiness Engineering ---o0o--- Department of automatic control REVIEWER’S COMMENT SHEET Student: Ta Tran Nhat Minh Student id: 19151034 Class: 191512A Major: Automation and Control Engineering Technology Project: Research, design and construct an autonomous golf-cart based on multi- task model and GPS navigation Advisor Assoc. Le My Ha COMMENT 1. Thesis’s content: The content of this thesis is good and novel. Advantage: The results of this thesis have been deployed in the real car and it can operate successfully.
Propose defending thesis: Approved 5. Overall evaluation: Very good 6.5 (In words:) Nine point five 7. Question: Please explain the method to detect the moving line at each corner of the road. Ho Chi Minh, June 2023 REVIEWER (Sign with full name) iv A GUARANTEE This thesis is the result of my study and implementation, which I hereby formally proclaim.
I did not plagiarize from a published article without author acceptance. I will take full responsibility for any violations that may have occurred. Ho Chi Minh, June 2023 AUTHOR (Sign with full name) Ta Tran Nhat Minh v ACKNOWLEDGMENT I would like to express my deepest grateful towards Assoc. Le My Ha for his inspiration, guidance, and instruction to support me for completing my graduation thesis.
During the whole progress, he not only helped me in finding solutions but also motivated and encouraged me to be able to successfully complete this project. Furthermore, I would like to express my sincere thanks to all lectures in Department of automatic control due to the accumulation of basic knowledge and experience they gave me in the four years of university. Especially, I also spent my honor to Faculty of Electrical and Electronics Engineering, for providing me the golf cart hardware for the research. In addition, I would like to thank Intelligent System Laboratory (ISLab) of Faculty of Electrical and Electronics Engineering and ISLab’s leader Phan Thanh Danh with ISLab’s car team, for the support of facilities and sharing their valuable experiences with me.
I also would like to express my gratitude to all ISLab members who strongly supported me when I dealt with this thesis. Finally, I take this opportunity to respect my parents and family for the encouragement, support, and attention throughout the project. Sincere thanks for everything! Ho Chi Minh, June 2023 AUTHOR (Sign with full name) Ta Tran Nhat Minh vi ABSTRACT In this thesis, an autonomous car utilizes a fusion between Multitask model, which includes scene parsing head and lane detection head, and multi-sensors data to manage a robust self-operating in the HCMUTE campus. The proposed multi-sensors comprise a Camera, GPS, and IMU (Inertial Measurement Unit) to operate my system.
In more detail, there are two major tasks for my project: Designing a controller board for steering control and enhancing self-driving car’s perception. To begin with, the microprocessor used as the centre element is STM32 and many on- board electronic components. The main functions of the mentioned board are controlling the steering angle and the car’s speed. Move to perception tasks, in this thesis, a fast and accurate multi-task approach for road line detection was proposed.
Despite the effectiveness of lane detection, the road line algorithm still suffers from hurdles on the street such as cars or people. To this end, I proposed an additional segmentation-based scene parsing module to work in parallel with the road line algorithm. The scene parsing acts as external guidance on where to make lane predictions on for the road line algorithm. The fusion of two methods was evaluated on a small driving scene dataset collected from a prototype autonomous car.
The experiment results illustrate that my system performs well on HCMUTE campus roads, with the accuracy of lane line detection is 93.50% and the mIOU value for Segmentation head is 0. As for the GPS navigation, the street map was first built up from three different GPS sensors data. After that, the was applied to reduce the noises from the environment and enhance the real-time performance of GPS modules. My work was implemented on my laptop with NVIDIA GTX 1050Ti graphic card as the primary processing unit.
Moreover, to reduce the execution burden, I inserted Jetson TX2 as a support processing unit for the mission of dealing with GPS and IMU data. Experimental results reveal that the whole system works well on several campus roads. The proposed method was applied to a prototype golf-based autonomous car, achieving accurate navigation results on campus road lanes, with about 34 FPS respectively. This result satisfies real-time practical applications.
vii CONTENTS BIÊN BẢN CHỈNH SỬA ĐATN HK2 NH 2022-2023. ii ADVISOR’S COMMENT SHEET. iii REVIEWER’S COMMENT SHEET. viii LIST OF FIGURES.
xii LIST OF TABLES .xv LIST OF ABBREVIATIONS. xvi CHAPTER 1: OVERVIEW .5 CHAPTER 2: LITERATURE REVIEW .1 Self-driving car technologies .2 Overview about Artificial Intelligence .2 Convolutional Neural Network .3 Image Segmentation Techniques .5 Extend Kalman Filter .10 Random Sample Consensus .31 CHAPTER 3: ASSEMBLE THE HARDWARE .1 Ezgo Golf Cart .3 Transmitter Devo 7 and Receiver RX-601 .8 Controller circuit board .9 External GPS module .55 CHAPTER 4: DESIGN AND CALCULATION .1 Multi-task road lane detection .2 Lane detection head .4 Scene parsing head .5 Total loss function .6 Enhance Scene parsing head .1 Steering wheel strategy .2 Multitask Lateral Controller.1 Building Campus Street map .2 Refined data map.3 Extend Kalman filter .4 Circular position checking.4 Fusion Strategy between Multitask model and GPS data .79 CHAPTER 5: EXPERIMENTS AND RESULTS.3 Results and comparisons .2 Evaluation supportive module .4 Steering wheel strategy performance .91 CHAPTER 6: CONCLUSION AND FUTURE WORK .98 xi LIST OF FIGURES Figure 1.1 Estimated spending in Self-driving R&D (2019) .2 Vin-AI autonomous car .3 The RC car model of HCMC UTE students .1 Modern driverless camera performance .2 GPS tracking system .3 IMU detailed components .4 IMU applications in mobile phone and in plane.5 Relation between Deep Learning, Machine Learning and AI .6 Structure of a biological neuron .7 Structure of artificial neurons .8 Compare performance between Machine Learning and Deep Learning .11 The result of a convolution operation .12 Curves of the Sigmoid, Tanh, and ReLU .13 Convolution with stride of 2 .15 Max pooling process .16 Fully connected layer .17 Three types of Image segmentation .18 The Residual block .19 Performance of Residual block with different layers testing .22 MTL using shared feature extractor .23 MTL using task-specific head .25 PyTorch Deep Learning framework .1 System’s block diagram .2 The Ezgo golf cart .3 Types of servo.4 Servo motor structure .5 Ezi Servo and driver .9 Steering unit mechanical.10 Steering unit in detail .17 Module GPS GNSS LC86L Quectel .22 Code flashing block .25 Distance sensor modules .27 Module control PWM of servo .28 Module control direction of servo .29 Module control car’s speed .31 3D PCB circuit board.32 Circuit board in final .33 Module GPS Ublox M8N .34 Module GPS beitian bn-880 .35 The connection between relays and actuators to control speed .36 The connected diagram between devices .1 Multi-threading in proposed system .2 Overall software design .3 Multi-task architecture .4 Lane detection head .5 Scene parsing head .6 Convolutional Block Attention Module with two submodules .7 Channel Attention Module .8 Spatial Attention Module .9 Multitask network with CBAM and CSP .10 The shortcoming of single lane model .12 Detail parameter in proposed car model .13 Calculate offset in Lane keeping scenario .14 Average offset point from segmented image .15 Block diagram of PID controller for vehicle angle in continuous time. Block diagram of PID for steering angle .17 The campus road map .18 Taylor series estimation .19 Block diagram of real time EKF.20 Circular checking position .2 My custom datasets.3 The data labeling process. The labeled images (a,c) and the collected images (b,d) .4 Labeling process using LabelMe tool .5 Raw data visualization .6 Training loss graph .