SIP15IE1SIE15IE15I215IE15IC/SIEISIE)GIE1GI21SIEISICISIEISIP/SIEIS]E2SIEISIEISICISIC/ĐIEISIEIE es | 5 MINISTRY OF EDUCATION AND TRAINING i HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION i E FACULTY FOR HIGH QUALITY TRAINING h | ] BỊ : Ũ U ñ iH dl iB rl fl a el fy al all Fj GRADUATION PROJECT fl : COMPUTER ENGINEERING TECHNOLOGY ñ io al fl H iH l i RESEARCH, DESIGN AND CONSTRUCT dl AN AUTONOMOUS GOLF CART : EF USING MULTISENSOR FUSION a Dị DỤ a H fal il ; ; h al ADVISOR: DR. LE MI HA 4) FT STUDENT: PHAN THANH DANH DỊ A NGUYEN TAN THIEN NIEN _| i h n DỤ i Fi E ; : A SKL009830 i if Ur = rl h : ty Ho Chỉ Minh city, January 2022 lì I| II " | i EIDIISIIEIEIDIIEISIEISIEIEIGIGIEIEIOIEIEISIDIGIEISIOIEIGISIOIEIEIEIOISEIEIOISEIEIDISIGISIEISIGIEIDLEISIOIEIIEIDIEIEISIOID HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING HCMUTE GRADUATION PROJECT RESEARCH, DESIGN AND CONSTRUCT AN AUTONOMOUS GOLF CART USING MULTISENSOR FUSION PHAN THANH DANH - 18119214 NGUYEN TAN THIEN NIEN - 18119033 MAJOR: COMPUTER ENGINEERING TECHNOLOGY ADVISOR: LE MY HA, Assoc.Prof Ho Chi Minh City, January 2022 HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING HCMUTE GRADUATION PROJECT RESEARCH, DESIGN AND CONSTRUCT AN AUTONOMOUS GOLF CART USING MULTISENSOR FUSION PHAN THANH DANH - 18119214 NGUYEN TAN THIEN NIEN - 18119033 MAJOR: COMPUTER ENGINEERING TECHNOLOGY ADVISOR: LE MY HA, Assoc.Prof Ho Chi Minh City, January 2022 an THE SOCIALIST REPUBLIC OF VIETNAM / is S Independence — Freedom— Happiness § By l ÁP %, về: Ho Chi Minh City, July 27, 2022 LO HIGH ques YY GRADUATION PROJECT ASSIGNMENT Student name: Phan Thanh Danh Student ID: 18119214 Student name: Nguyén Tan Thién Nién Student ID: 18119033 Major: Computer Engineering Technology Class: 18119CLA1 Advisor: Assoc. Prof Lé My Ha Phone number: 0938811201 Date of assignment: Date of submission: 1. Project title: Research, design, and construct an autonomous golf cart using multisensor fusion 2.
Initial materials provided by the advisor: - Image processing and machine learning documents such as papers and books: - The related thesis of previous students - The hardware specifications and its review. Content of the project: - Refer to documents, survey, read and summarize to determine the project directions - Calculate parameters and design block diagram for steering system using DC servo. - Try and handle errors in the wheeling system (mechanical and electrical) - Collect and visualize data of sensors - Choose models and algorithms for the car’s perception - Write programs for microcontrollers and processors - Test and evaluate the completing system - Write a report - Prepare slides for presenting 4. Final product: The golf cart model uses a multisensor combination that has two modes: Automatic and Manual.
The golf cart can operate well on HCMUTE campus with not many complex scenarios. CHAIR OF THE PROGRAM ADVISOR (Sign with full name) (Sign with full name) Brenig. THE SOCIALIST REPUBLIC OF VIETNAM đŸ rN Independence — Freedom— Happiness l; Bn $ về Ho Chi Minh City, July 27, 2022 ` sey ¬Š #WGH QUNỀ + ADVISOR’S EVALUATION SHEET Student name: Phan Thanh Danh Student ID: 18119214 Student name: Nguyén Tan Thién Nién Student ID: 18119033 Major: Computer Engineering Technology Project title: Research, design, and construct an autonomous golf cart using multisensor fusion Advisor: Assoc. Prof L¢é My Ha EVALUATION 1.
Content of the project: - The thesis has a total of six chapters with 73 pages - The construction and design of an autonomous golf cart that can run on the HCMUTE campus with two modes: Manual and Automatic - The system works with a combination of different sensors and their according algorithms. - The real system is successfully completed following the objectives in the proposal. - The thesis forms a basic foundation for the next generation of HCMUTE students in the field of the practical autonomous car. Strengths: - The system can support the UTE students and lectures in moving around campus - The golf car is designed with image processing and machine learning algorithms combined with the control technique and mechanism design.
- The whole sensors of this project are low-cost. - The execution time for the automatic mode is suitable for practical application. - The accuracy and the safety of the system are guaranteed. Weaknesses: - The system cannot operate fully autonomous without an operator, because several things need to be upgraded and improved.
- All the sensors and hardware are low-cost. Therefore, the accuracy and the effectiveness of the system are just acceptable in real environments. Approval for oral defense? (Approved or denied) Approved 5. Overall evaluation: (Excellent, Good, Fair, Poor) Excellent 6.
Mark: 10 (In words: Ten) Ho Chi Minh City, July 27, 2022 ADVISOR (Sign with full name) 1 APPENDIX 5: (Pre-Defense Evaluation sheet) VP Gan THE SOCIALIST REPUBLIC OF VIETNAM / Ÿ 3 s\ Independence — Freedom— Happiness Jx * $ TT nnuaaaaaa “ MS “tp an ‘ Ho Chi Minh City, January 20, 2020 PRE-DEFENSE EVALUATION SHEET Student name: Phan Thanh Danh Student ID: 18119214 Student name: Nguyén Tan Thién Nién Student ID: 18119033 Major: Computer Engineering Technology Project title: Research, design, and construct an autonomous golf cart using multisensor fusion Name up cà¿à Tố. Content and workload of the project 6.ÔỎ ) Ho Chi Minh City, month day, year REVIEWER (Sign with full name) 11 APPENDIX 6: (Evaluation sheet of Defense Committee Member) THE SOCIALIST REPUBLIC OF VIETNAM Independence — Freedom— Happiness EVALUATION SHEET OF DEFENSE COMMITTEE MEMBER Student name: Phan Thanh Danh Student ID: 18119214 Student name: Nguyén Tan Thién Nién Student ID: 18119033 Major: Computer Engineering Technology Project title: Research, design, and construct an autonomous golf cart using multisensor fusion Name of Defense Committee Member: EVALUATION 1. Content and workload of the project 4. ) Ho Chỉ Minh City, month day, year COMMITTEE MEMBER (Sign with full name) iv ACKNOWLEDGEMENT We would like to sincerely thank Professor Le My Ha for his thorough instruction, which helped us to have the necessary information to use for completing the thesis.
During the whole progress, even if we did our best to complete everything completely, mistakes are still inevitable. We anticipate having my advisor's focused assistance and direction to help us gain more experience and successfully complete the topic project. On the other hand, we would like to express our sincere thanks to the Faculty of Hight Quality Training and Faculty of Electrical and Electronics Engineering where we obtained basic knowledge and experience. Especially, we received the golf cart from FEEE honorably.
Moreover, we also would like to thank ISLab members who helps us in detailing the works of this project. They shared valuable experience and knowledge with us. Ultimately, we would like to express our gratitude to our families for their support of our team throughout the implementation of this thesis. Sincere thanks for everything! A GUARANTEE This thesis is the result of our study and implementation, which we hereby formally proclaim.
We did not plagiarize from a published article without author acceptance. We will take full responsibility for any violations that may have occurred. Authors Phan Thanh Danh Nguyễn Tắn Thiên Niên vi ABSTRACT Autonomous cars will be able to make an analysis and manage themselves on the ongoing path, depending on scene understanding and surrounding observation. Particularly, the automobile does need to clarify the whole information surrounding it.
Inspired by that ideas, in this paper, we proposed a multi-sensor fusion method for autonomous cars operating on the HCMUTE campus. The fusion method comprises three types of sensors, which are a Camera, GPS, and 2D LiDAR. To begin with, we utilized and enhanced two deep learning models, which are lane-line detection and semantic segmentation. Two of these models are pre-trained and fine-tuned on our self-labeled dataset.
As for the GP signal, we used Kalman Filter to reduce the noises from the environment and then check the continuous destination by a circular equation. Additionally, we took advantage of 2D LiDAR as the safety term during the car avoidance process. Last but not least, we combined all these programs by using threading and distributed system, which communicates with each other by User Datagram Protocol. The system uses our laptop with NVIDIA GTX 1650 graphic card as the primary processing unit.
To reduce the execution burden of our laptop, we subjoined a Jetson TX2 for processing GPS and LiDAR sensors. In terms of the controlling system, the main microcontroller is two Arduino boards (Mega and Nano); one is for steering control, and another is for speed control. Furthermore, we designed a simple interface using PyQT to display the on-road information. Experimental results reveal that the whole system works well on several campus roads.
Furthermore, the lowest frame per second of the system is 20, which satisfies real- time practical applications. vii Contents ACKNOWLEDGEMENT. Sàn HH HH HH. „ VỈÏ LIST OF FIGURES .X LIST OE TABLES.1 OVERVIEW AND RELATED RESEARCH.1 Convolutional Neural Network 2.3 Common backbones on BBR RN NHB ED 2.1 Point cloud clustering.---- -- 5 Sàn HH HH HH HH HH HH 14 V 5vi)009)/9040)000.
0 ốố ốốố ốố ố. HARDWARE PLUA TEFORM. ne ener 2H HH. LH HH HH HT HT 20 K58) v090 7 .1 Ezgo Golf CAF.
(1 12 11T HH HH TH HT HH TT HT 21 CHAPTER 4. HH HH HH HH HH 33 4.1 RGB IMAGE-BASED ALGORITHMS.1 Enhanced Semantic Segmenfafi0n.2 Lane-Line Deftection. ---- -- 1S 12H HH HH HH HH HT HH HH Hiệp 36 4.2 ALGORITHMS OF GPS DA TTA.“ HH HH HH ghế 37 Vili 4.3 ALGORITHMS ON 2D LiDAR .4 FUSION OF CAMERA AND 2D LIDAR.5 ESTIMATION AND CONTROL OF STEERING ANGLE .6 WHEELING STRA TEGY. renee reer HH HH HH 44 CHAPTER 5.
----- ¿+ LH HH HH HH HH TH TH HH Hit 47 11).4 COMPARISONS AND EVALUATION CHAPTER 6. CONCLUSION AND FUTURE WORK. ix LIST OF FIGURES Figure 2. Basic CNN architeCture: [ ].- --- - + + 12121 91 91 1 1191 1 11v ng nen 4 Figure 2.
The difference between three types of segmentation.The residual bloCK. Performance of Residual block with different layers testing. Resnet-18 archi(€C{UTE.---- «6 + tk HH gi, weed Figure 2. Depthwise convolution, uses 3 kernels to transform a 12x12x3 image to an 8x83 T8 .- HH TH TH HT TT HT TH TT TT HT TH TT TH TH 8 Figure 2.
Pointwise convolution, transforms an image of 3 channels into an image of 1 i00 ốốốốốốốốố ốốố ốố. Pointwise convolution with 256 kernels, outputting an image with 256 i00 Tố ốốố ốố. Optimizer algorithms deveÏODIm€HI(.------ - - << %1 k*1E 91 1 91 1 1 1 1 1 11T ng ngờ 10 Figure 2. Adaptive Breakpoint Detection algorithm.- 5 «+ s+sx+s£sersezsesees 12 Figure 2.- - 6 << 111 1S 911111101 HH ngàn HH nhân 13 Figure 2.
--¿- - << + ** k1 E1 1E E1 1 11111 11T HH ngàn ghế 15 Figure 2. Function and Operation Of PyfOFCH.- ¿+ St Sk SE rưn 16 Figure 2. (6 1S 91 1E E1 1 1 1 1 11101 HH ghi 17 Figure 2.- ‹ -¿- ¿+ Sàn TT HH HH TH Hành 18 Figure 2. Multithreading for the application oo.
cece cece eeceseteeseseeeeseseteeaees 19 Figure 3. 1: The overall hardware platform Figure 3. The EZGO golf cart. EZ1 S€TVO and (TIV€T.-- - «6 1h 1T TT HT TH Tnhh Hân Figure 3.
¿E11 E1 1E E111 11T HH TH Hàn răn Figure 3. HT HH TH TT HH TT TH TT TT HH HH 25 Figure 3. RP-Lidar AL occ. Module GPS Ublox M8N .- 6 LH HH rà 26 Figure 3.
Waveshare touch SŠCT€€II.-- ¿6+1 11T TT HT HH ng Hàn 27 Figure 3. Module WIFI TOU€T.- 6 << 1S 1k E111 1T TT TH Tàn HH nhăn 28 Figure 3. Laptop Acer Nitro 5 .- -- --<k HH TH TH TH TT Hàn răng 29 Figure 3.