MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING GRADUATION THESIS COMPUTER ENGINEERING TECHNOLOGY DESIGN AND IMPLEMENTATION OF A SELF-DRIVING CAR FOR FOLLOWING LANE ADVISOR: ASSOC. TRUONG NGOC SON STUDENTS: LE THI KIEU GIANG NGUYEN HUNG THINH SKL011174 Ho Chi Minh City, June 2023 HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING GRADUATION PROJECT DESIGN AND IMPLEMENTATION OF A SELF-DRIVING CAR FOR FOLLOWING LANE LÊ THỊ KIỀU GIANG Student ID: 19119001 NGUYỄN HƯNG THỊNH Student ID: 19119056 Major: COMPUTER ENGINEERING TECHNOLOGY Advisor: TRƯƠNG NGỌC SƠN, Assoc. Ho Chi Minh City, June 2023 THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- Ho Chi Minh City, June 26, 2023 GRADUATION PROJECT ASSIGNMENT Student name: Lê Thị Kiều Giang Student ID: 19119001 Student name: Nguyễn Hưng Thịnh Student ID: 19119056 Major: Computer Engineering Technology Class: 19119CLA Advisor: Assoc. Trương Ngọc Sơn Phone number: 0931085929 Date of assignment: 25 February, 2023 Date of submission: 26 June, 2023 1.
Project title: Design and implementation of a self-driving car for following lane 2. Initial materials provided by the advisor: Documents such as paper and books related to AI, control hardware devices. Content of the project: - Analyze the challenges of the project. - Learn about the technical specifications, guiding thought and theoretical basis of the components of the hardware.
- Propose the model and summarize the overall system. Design block diagram, principal diagram. - Pre-processing data (cleaning data, generating object detection data). - System configuration and design hardware.
- Test run, check, evaluate and adjust. - Conduct report writing. Final product: A car model that moves along the lane and can recognize traffic signs, a final report, a video. CHAIR OF THE PROGRAM ADVISOR (Sign with full name) (Sign with full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- Ho Chi Minh City, June 26, 2023 ADVISOR’S EVALUATION SHEET Student name: Lê Thị Kiều Giang Student ID: 19119001 Student name: Nguyễn Hưng Thịnh Student ID: 19119056 Major: Computer Engineering Technology Project title: Design and implementation of a self-driving car for following lane Advisor: Assoc.
Trương Ngọc Sơn EVALUATION 1. Content of the project:. Approval for oral defense? (Approved or denied). ) Ho Chi Minh City, June …, 2023 ADVISOR (Sign with full name) ĐẠI HỌC SƯ PHẠM KỸ THUẬT CỘNG HOÀ XÃ HỘI CHỦ NGHĨA VIỆT NAM THÀNH PHỐ HỒ CHÍ MINH Độc lập – Tự Do – Hạnh phúc KHOA ĐÀO TẠO CHẤT LƯỢNG CAO Tp.
HCM, ngày 15 tháng 07 năm 2023 BẢN GIẢI TRÌNH CHỈNH SỬA ĐỒ ÁN TỐT NGHIỆP NGÀNH: CNKT MÁY TÍNH 1. Tên đề tài: Design and implementation of a self-driving car for following lane 2. Tên sinh viên: Lê Thị Kiều Giang MSSV: 19119001 Tên sinh viên: Nguyễn Hưng Thịnh MSSV: 19119056 3. Trương Ngọc Sơn 4.
Hội đồng bảo vệ HĐ 1, phòng A4-401, ngày 07 tháng 07 năm 2023 5. Giải trình chỉnh sửa báo cáo đồ án tốt nghiệp: TT Nội dung góp ý của Hội đồng Kết quả chỉnh sửa, bổ sung Ghi chú Đã bổ sung “AI system will The authors have mentioned transmit signals to the control that “AI system will transmit system, including angle and speed signals to the control system, data. The angle data will be 1 including angle and speed calculated through the PID data”. Which function method and the speed data will be calculates these parameters in obtained from the Linear this system? function.
Đã kiểm tra và chỉnh sửa, kết quả Authors should be replace “the 2 trình bày được bổ sung trong group”, “our team” by “we” trang 1, 2, 3, 19 và 65. Authors should shorten the thesis objectives, focus on the Đã kiểm tra và chỉnh sửa, kết quả feature or implementation 3 trình bày được bổ sung trong phần methods of your “self-driving 1. car for following lane” instead of researching theory or kits. 4 Chapter 3 Design and Đã kiểm tra và chỉnh sửa, kết quả Số hiệu: BM16/QT-PKHCN-QHQT-NCKH/02 Lần soát xét: 02 Ngày hiệu lực: 01/4/2020 Trang: 1/2 Implementation: authors trình bày được bổ sung trong phần should not introduce too much 3.
about board or kit specification; instead, focus on the requirements of blocks and your “methods” to process or “designs” of blocks to meet these requirements.1: The name of blocks should be “DC motor Đã kiểm tra và chỉnh sửa, kết quả control”, “Servo motor 5 trình bày được bổ sung trong control” instead of “Control Figure 3. motor DC” and “Control Servo”. Đã kiểm tra và chỉnh sửa, kết quả Flowchart: the “begin” and trình bày được bổ sung trong “end” should be represented in Figure 3.7, Figure 6 terminator shapes instead of 3. Xác nhận của trưởng Xác nhận của GVHD Nhóm thực hiện báo cáo ngành (Ký họ và tên) (Ký họ và tên) (Ký họ và tên) Số hiệu: BM16/QT-PKHCN-QHQT-NCKH/02 Lần soát xét: 02 Ngày hiệu lực: 01/4/2020 Trang: 2/2 DECLARATION We hereby declare that this is the final report, "Design and implementation of a self-driving car for following lane".
The simulations and study findings are accurate and were carried out entirely under the direction of the instructor, Assoc. TRUONG NGOC SON. The report does not duplicate any other sources either. Additionally, the paper includes a variety of cited and carefully labeled reference materials.
Before the department, faculty, and school, we would like to fully accept responsibility for this promise. Student LE THI KIEU GIANG NGUYEN HUNG THINH i ACKNOWLEDGEMENTS First, we would like to express my deepest gratitude to the School Board of the Ho Chi Minh City University of Technology and Education, as well as the Faculty for High Quality Training, for establishing wonderful conditions for me to pursue our project. In addition, we would also like to express our sincere thanks to the Head of the Department, Assoc. Truong Ngoc Son, who always closely follows the learning situation and encourages and creates development opportunities for each generation of students.
Last but not least, we cannot prevent mistakes due to a lack of knowledge and implementation time. We welcome your feedback and suggestions to help us improve this topic. Many thanks for all your help and regards. Student LE THI KIEU GIANG NGUYEN HUNG THINH ii TABLE OF CONTENTS DECLARATION .ii TABLE OF CONTENTS.
iii LIST OF FIGURES. v LIST OF TABLES .viii LIST OF ABBREVIATIONS. Object and Scope of the study. Object of the study.
Scope of the study. 3 CHAPTER 2:LITERATURE REVIEW. Overview of CNN. Theory U-net architecture.
Creating layers in U-net architecture. The overview of a self-driving car. Other techniques used in the Project. Pulse width modulation (PWM) .17 CHAPTER 3: DESIGN AND IMPLEMENTATION.
Requirements of the topic. 46 CHAPTER 4: EXPERIMENT RESULTS AND DISCUSSION. Sign traffic generation. Evaluation and comparison .500 dataset and Extra 2000 dataset.
Evaluate of the Yolo model. FPS expansion and improvement. Run a Yolov4 and YoloV5 model by converting it to the TensorRT engine and then running it in Jetson Nano. Improve the detection accuracy of Yolo.
61 CHAPTER 5: CONCLUSION AND FUTURE WORKS. 69 iv LIST OF FIGURES Figure 2.1: Architecture of CNN network .2: YOLO network architecture diagram.3: Types of Image Segmentation. Each bluebox corresponds to a multi-channel feature map.5: Layers in U-net architecture .8: Transmitted in the form of packets of I2C .10: Transmitted in the form of packets of UART .1: Block of Design a self-driving car for following lane system .2: The detailed block of the AI system .3: NVIDIA Jetson Nano Developer Kit eMMC .4: Camera Raspberry Pi V2 IMX219 8MP .5: The schematic of the AI system .6: The flowchart of recognized lanes .7: The flowchart of generate the lane .8: The process flow of Labeling and classification for lane dataset .9: Result the name of class .10: The process flow of Labeling and classification for traffic signs dataset .11: Format to export dataset .12: Detection of images .13: Parameter of ratio Frame .14: The process flow of Train Yolv8 on Google Colab .15: Create CNN network .16: The flowchart of Recognize CNN network .17: The flowchart of Convert model .18: The flowchart of Training U-net Segmentation processing .19: Processing for Training U-net Segmentation .20: The process flow of Detecting traffic signs in real-time .21: The flowchart of Detecting Lane in real-time .25: Bounding box of traffic sign .26: The detailed block of the Control system .28: H-Bridge Motor driver L298N .30: H-Bridge Pin Configuration .32: PCA9685 16-Channel PWM Driver .33: RC Servo MG996R Motor .34: 18650 Li-Ion Rechargeable Battery 3.35: The schematic of the Control system .36: The flowchart of DC Motor Control .37: The flowchart of Servo Motor Control .1: Traffic map of the self-driving car system .2: The structure layers of the self-driving car system .3: The recognition of lane and traffic signs by autonomous vehicles .4: The recognition of lane by autonomous vehicles .5: The recognition of traffic signs by autonomous vehicles .6: Result of generating lane process .8: Result of labelling each lane image .9: The segmentation of image .10: Result of generating Sign traffic process .11: Traffic sign details .12: Result of labelling each traffic sign image .13: Labelling of other classes .14: Loss and Accuracy Graph of 2.500 dataset first with 100 epochs .15: Results of test image from 2.500 dataset first with 100 epochs .16: Loss and Accuracy Graph of extra 2000 dataset second - 100 epochs .17: Results of test image from extra 2000 dataset second - 100 epochs .18: Result of the recognized traffic sign by Yolo .19: Loss and Accuracy Graph of training with CNN model .20: The process of converting the Yolo models into the TensorRT engine .21: Result of run the command $ cmake .22: Result of run the command $ make .23: Result of export to yolov5n.24: The flowchart of Deploy engine file on Jetson Nano by TensorRT .25: The flowchart of combining Yolo with Classification .26: The flowchart of combining Yolo without Classification .27: Results of the combination of Yolo, classification, and segmentation. 64 vii LIST OF TABLES Table 3.1: Camera Raspberry Pi V2 IMX219 8MP Specifications .2: 12V gear motor Specifications .3: RC Servo MG996R Motor Specifications .1: Results of training for Lane detection model .2: Results of deploying for Lane detection model .3: Results of training for Yolo versions .4: Results of deploying for Yolo versions .5: Results of Classification .6: Results of deploying detection lane and traffic signs.
63 viii LIST OF ABBREVIATIONS AI Artificial intelligence CV Computer vision CNN Convolutional Neural Network YOLO You Only Look Once PWM Pulse width modulation I2C Inter-Integrated Circuit UART Universal Asynchronous Receiver/Transmitter FPS Frame per second I2C Inter-Integrated Circuit TWI Two Wired Interface SDA Serial Data SCL Serial Clock MSB Most significant bit UART Universal Asynchronous Receiver/Transmitter LSB Least significant bit PID Proportional-Integral-Derivative GPU Graphics processing unit CPU Central Processing Unit GPIO General Purpose Input/Output ix ABSTRACT Several businesses, including Tesla, Audi, Google, etc., are developing self-driving vehicles. The idea of autonomous cars originated from the occurrence of accidents due to careless people driving and, in this way, dangerous accidents frequently occur. We decided to implement the topic "Design and implementation of a self-driving car for following lane". Our project aims to add lane detection as a safety function to cars so that they will not divert attention from the road when the driver is not alert.
This can make operating an automobile stronger, simpler, and much safer than it was previously. We researched numerous papers and articles about it for this aim. It is interesting to see how many businesses and developers are contributing to this subject. What we did was combine innovative image processing methods with machine learning models.
The most difficult task was interpreting these lines with the image to make sense of the lanes.