MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION GRADUATION THESIS AUTOMATION AND CONTROL ENGINEERING TECHNOLOGY APPLICATION OF DELTA ROBOT IN A PRODUCT CLASSIFICATION SYSTEM APPLYING IMAGE PROCESSING INSTRUCTOR: MR.TRAN MANH SON STUDENT: DUONG THANH DAT VO TIEN DAT SKL012366 Ho Chi Minh City, DECEMBER, 2023 MINISTRY OF EDUCATION & TRAINING HO CHI MINH UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF INTERNATIONAL EDUCATION GRADUATION PROJECT APPLICATION OF DELTA ROBOT IN A PRODUCT CLASSIFICATION SYSTEM APPLYING IMAGE PROCESSING Advisor: Tran Manh Son, Ph.D Students: Duong Thanh Dat 19151052 Vo Tien Dat 19151053 Major: Automation and Control Engineering Technology Year of Admisssion: 2019 – 2023 Ho Chi Minh City, December 2023 HCMC UNIVERSITY OF THE SOCIALIST REPUBLIC OF VIETNAM TECHNOLOGY AND EDUCATION Independence – Freedom – Hapiness Falcuty of International Education ----o0o---- Ho Chi Minh City, December 30, 2023 GRADUATION THESIS PROJECT TASKS Full name of student 1: Duong Thanh Dat Student ID: 19151052 Full name of student 2: Vo Tien Dat Student ID: 19151053 Instructor’s full name: Mr. Tran Manh Son Contact Number: 0919 197 139 Major: Automation and Control Engineering Technology Forms of training: Formal training Year of Admission: 2019 – 2023 Class: 19151CLA3 Date of delivery: 05/9/2023 Task complete date: 26/12/2023 I. THESIS NAME Application of Delta robot in a Production Classification System applying Image Processing II. ORIGINAL DOCUMENTS − Documentation of PLC controller, PLC positioning control manual, Arduino control.
− Documentation of servo motor, servo driver, step motor and driver. − Documentation of deep learning, YOLO model for training image base on Convolutional neural network and raw image processing using OpenCv library in python. CONTENT OF IMPLEMENTATION − Learn the fundamental theory of hardware components. − Enhance the hardware components for a Delta robot and address signal latency when transmitting data from Visual Studio Code to the middleware (KepserverEx6).
− Investigate and improve image processing techniques for detecting the center and rotation angle of objects. Page | I − Improve the formulas for converting object coordinates from a camera to robot coordinates. − Develop code for controlling robot motors using PLC software and regulate conveyor speed using Arduino IDE. − Set up the connection for transmitting and receiving signals between the conveyor belt and the PLC.
− In multiple iterations, validate and experiment with the accuracy of a Delta robot in classifying products, specifically square and circular objects with four different colors. Accuracy in this context encompasses the precision in gripping the object's center and ensuring the object rotates accurately (always aligning the object's rotation with the y-axis of the camera's coordinate system). PRESENTATION LANGUAGE − Report Document: English ☒ Vietnamese ☐ − Thesis Defense Presentation: English ☒ Vietnamese ☐ INSTRUCTORS AUTOMATIC CONTROL (Sign and write full name) DEPARTMENT (Signed & stamped) Page | II HCMC UNIVERSITY OF THE SOCIALIST REPUBLIC OF VIETNAM TECHNOLOGY AND EDUCATION Independence – Freedom – Hapiness Falcuty of International Education ----o0o---- Ho Chi Minh City, December 30, 2023 GRADUATION THESIS PROJECT SCHEDULE Full name of student 1: Duong Thanh Dat Student ID: 19151052 Full name of student 2: Vo Tien Dat Student ID: 19151053 Instructor’s full name: Mr. Tran Manh Son Contact Number: 0919 197 139 Major: Automation and Control Engineering Technology I.
THESIS NAME Application of Delta robot in a Production Classification System applying Image Processing II. PROJECT IMPLEMENTATION PLAN Instructor Week/ Date Content confirm - Get an overview of the project topic. - Research on Delta robots and practical applications of 05/09/2023 robots. (1 week) - Learn about servo and stepper motors and how to wire between driver and motor.
- Execute tests on the project using the previous programming framework and identify the encountered 12/09/2023 issues. (1 week) - Through multiple trial runs, propose solutions for existing errors and organize the implementation sequence of these solutions. - Improve image processing in using YOLO for shape 19/09/2023 and color detection and raw image processing for object (2 week) center and rotation angle determination. Page | III - Addressing the first system issue: Incorrect center gripping by the robot.
03/10/2023 (2 week) - Solve by adjusting the object coordinate calculation formula and minimizing signal latency when transmitting data from Python to KepServerEx6. - Determine the second system issue: Delta robot speed is 17/10/2023 too slow and inconsistent. (2 week) - Solve by completely change the PLC code. - Testing the 4th DoF of the robot (rotation angle) 31/10/2023 (2 week) - Fixing the 4th DoF error by changing the object’s angle programming in Python and in PLC.
14/11/2023 - Design to improve the robot’s system hardware: arms, (1 week) limit switch holder, 4th DoF step motor holder. - Developing a formula to adjust the robot's movement 21/11/2023 speed to synchronize with the conveyor belt speed. (2 week) - Testing the system with three different conveyor belt speeds. - Conduct test runs multiple times on different lighting 05/12/2023 conditions and different conveyor speeds.
(2 week) - Evaluate and compile statistics on system accuracy and performance on those test runs. - Write a graduation project report. 19/12/2023 (1 week) - Record a video about the project's operating process and complete the powerpoint. INSTRUCTORS (Sign and write full name) Page | IV HCMC UNIVERSITY OF THE SOCIALIST REPUBLIC OF VIETNAM TECHNOLOGY AND EDUCATION Independence – Freedom – Hapiness Falcuty of International Education ----o0o---- Ho Chi Minh City, December 30, 2023 ASSESSMENT FORM OF INSTRUCT LECTURER Full name of student 1: Duong Thanh Dat Student ID: 19151052 Full name of student 2: Vo Tien Dat Student ID: 19151053 Major: Automation and Control Engineering Technology Year of admission: 2019 – 2023 Class: 19151CLA3 Full name Instructor: Mr.
Tran Manh Son Thesis name: Application of Delta robot in a Production Classification System applying Image Processing COMMENT: 1. Regarding the topic content and implementation volume:. Recommend for defend graduation thesis or not?. ) INSTRUCTORS (Sign and write full name) Page | V HCMC UNIVERSITY OF THE SOCIALIST REPUBLIC OF VIETNAM TECHNOLOGY AND EDUCATION Independence – Freedom – Hapiness Falcuty of International Education ----o0o---- Ho Chi Minh City, December 30, 2023 ASSESSMENT FORM OF DECIPATION LECTURER Full name of student 1: Duong Thanh Dat Student ID: 19151052 Full name of student 2: Vo Tien Dat Student ID: 19151053 Major: Automation and Control Engineering Technology Year of admission: 2019 – 2023 Class: 19151CLA3 Full name Instructor: Mr.
Tran Manh Son Thesis name: Application of Delta robot in a Production Classification System applying Image Processing COMMENT: 1. Regarding the topic content and implementation volume:. Recommend for defend graduation thesis or not?. ) REVIEW LECTURER (Sign and write full name) Page | VI HCMC UNIVERSITY OF THE SOCIALIST REPUBLIC OF VIETNAM TECHNOLOGY AND EDUCATION Independence – Freedom – Hapiness Falcuty of International Education ----o0o---- Ho Chi Minh City, December 30, 2023 GUARANTEE We hereby commit that the research presented in this graduate thesis is done by ourselves.
Excerpts and images used in the thesis are attached to their references, which are made public following the rules. Project Executor Student 1 Student 2 Duong Thanh Dat Vo Tien Dat Page | VII ACKNOWLEDGEMENT After undergoing the process of completing the graduation project for over 3 months, our team has received generous assistance and support from the teachers at the Ho Chi Minh City University of Technology and Education, especially the dedicated guidance and valuable suggestions from Mr. Tran Manh Son, which greatly benefited our team throughout the project. We would like to express our deepest gratitude to Mr.
Tran Manh Son. Furthermore, during our 4-year academic journey at the Ho Chi Minh City University of Technology and Education, we also want to express our gratitude to the subject teachers in the Faculty of High-Quality Training and the Faculty of International Education for their dedication in teaching and warm guidance, which has helped all members of our team accumulate a wealth of knowledge and experience to apply to the project and beyond. We are very grateful. Given our limited time and understanding, our team faced various challenges while delving into the research for this project, resulting in inevitable shortcomings.
We sincerely hope to receive feedback from our teachers to enhance the project to the best of our ability. In conclusion, our team wishes the teachers good health to continue their teaching journey. Once again, we sincerely appreciate! Ho Chi Minh City, December 2023 Executed student (Sign and write full name) Page | VIII SUMMARY After completing the graduation project on the topic "Application of Delta Robot in a Production Classification System applying Image Processing," our team has acquired additional knowledge in robotics, particularly Delta robots, as well as fundamental and advanced image processing techniques. In this report, we will outline the steps taken by our team to accomplish this thesis.
Firstly, we enhanced the image processing capabilities of the Delta robot by improving the quality of input images for the YOLO model. This was done to enhance the quality of object detection by the robot and improve the accuracy of object center and rotation angle identification in simple image processing. Subsequently, our team designed and improved certain hardware components to make the Delta robot more flexible and robust in its operations. Following this, we refined the position calculation formulas to precisely control the Delta robot's product classification and simultaneously improved data processing speed and robot movement speed to their maximum allowable limits while maintaining accuracy in the product classification process.
Finally, our team conducted tests and evaluations on the classification results of four types of objects (rectangular and circular objects in white, rectangular objects in three different colors, circular objects in three different colors) under various lighting conditions and different conveyor belt speeds. We conclude with a final assessment of the accuracy and processing speed of the system. Page | IX TABLE OF CONTENTS GRADUATION THESIS PROJECT TASKS .I GRADUATION THESIS PROJECT SCHEDULE. III ASSESSMENT FORM OF INSTRUCT LECTURER.
V ASSESSMENT FORM OF DECIPATION LECTURER. IX TABLE OF CONTENTS. X LIST OF FIGURES. XIII LIST OF TABLES.1 Reason for choosing the topic .3 Object of research scope .4 Approach and research methods.
3 CHAPTER 2: THEORETICAL BASIS .1 Introduction to Delta robot .1 Structural principle of Delta Robot .2 Working principle of Delta Robot .2 Introduction to PLC .3 Introduction to image processing .1 Concept of raw image processing.4 Overview of CNN, YOLO Model and Software Used for Network Training .1 An Overview of Convolutional Neural Networks (CNN).4 Sofware used for training process .6 Visual Studio Code. 21 CHAPTER 3: HARDWARE DESIGN AND IMPLEMENTATION .2 Design the Delta robot model in Solidworks .2 End – effector platform .5 Limit switch holder.1 PLC Mitsubishi FX5U .3 AC servo motor .7 Camera XIAOVV HD USB .9 Pneumatic actuation equipment .11 Others electrical device .1 Pratical Delta robot model .3 Electrical circuit diagram. 44 Page | XI CHAPTER 4: SOFTWARE CALCULATION AND DESIGN .1 Delta robot kinematic equation .1 Delta robot construction .2 System operation procedure and Algorithm flowchart .3 The process of object detection, classification and positional computation using image processing .1 Object detection and classification using YOLO model .2 Detecting the coordinates of the object's center and the object's orientation .4 Method of converting frame coordinate to robot system coordinate .5 Calculate the robot's velocity value to pick up the correct coordinates of the object's center: .6 Programming and design of control interface for robot system. 66 CHAPTER 5: VERIFICATION AND EVALUATION OF PRATICAL PERFORMANCE OF THE SYSTEM .1 Verificatiom of the object classification process .1 Classification result of 8 object .2 Evaluation of the object classification process.2 Evaluating the overall system’quality .3 Some error cases, reason and solutions.
76 CHAPTER 6: CONCLUSION AND FUTURE DEVELOPMENT. 82 Page | XII LIST OF FIGURES Figure 2. 1: A Delta robot model from Topstar company. 2 The Image Processing procedure .3: Grayscale image levels .4: RBG color space .5: The color range represent the HSV color system .6: Thresholding image with various algorithm .