MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION GRADUATION THESIS MAJOR: AUTOMATION AND CONTROL ENGINEERING TECHNOLOGY THE AUTOMATIC LABELLING CONTROL AND MONITORING SYSTEM INSTRUCTOR: NGO VAN THUYEN STUDENT: PHAM HONG DANG PHUNG NGUYEN VIET HUNG Ho Chi Minh city, June 2024 MINISTRY OF EDUCATION & TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION INTERNATIONAL FACULTY AUTOMATIC CONTROL DEPARTMENT -----------------⸙∆⸙----------------- GRADUATION THESIS PROJECT: THE AUTOMATIC LABELLING CONTROL AND MONITORING SYSTEM INSTRUCTOR: Associate Professor PhD. NGO VAN THUYEN STUDENT 1: PHAM HONG DANG STUDENT ID: 20151019 CLASS: 20151CLA2 STUDENT 2: PHUNG NGUYEN VIET HUNG STUDENT ID: 20151281 CLASS: 20151CLA4 HO CHI MINH CITY, June 2024 MINISTRY OF EDUCATION & TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION INTERNATIONAL FACULTY AUTOMATIC CONTROL DEPARTMENT -----------------⸙∆⸙----------------- GRADUATION THESIS PROJECT: THE AUTOMATIC LABELLING CONTROL AND MONITORING SYSTEM INSTRUCTOR: Associate Professor, PhD. NGO VAN THUYEN STUDENT 1: PHAM HONG DANG STUDENT ID: 20151019 CLASS: 20151CLA2 STUDENT 2: PHUNG NGUYEN VIET HUNG STUDENT ID: 20151281 CLASS: 20151CLA4 HO CHI MINH CITY, June 2024 THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- Ho Chi Minh City, June 25th, 2024 GRADUATION PROJECT ASSIGNMENT Student name: Phạm Hồng Đăng Student ID: 20151091 Class: 20151CLA2 Student name: Phùng Nguyễn Việt Hùng Student ID: 20151281 Class: 20151CLA4 Major: Automation and Control Engineering Technology Supervisor: Associate Ph.D Ngô Văn Thuyên Phone number: _________________ Date of assignment: _____________________ Date of submission: _____________ 1. Project title: The Automatic Labelling Control and Monitoring System.
Initial materials provided by supervisor: ___________________________________ 3. Content of the project: − Analyse Mitsubishi PLC − Examine the functionalities of RobotStudio simulation tools. − Analyse vision problems: Optical Character Recognition and camera calibration. − Select and design a communication architecture for system integration.
− Run and calibrate software and hardware. − Evaluate performance results and write a thesis. Final product: − 01 Project report documents. − Complete model of automatic labelling control and monitoring system according to original design requirements.
− Management interface according to original design requirements. CHAIR OF THE PROGRAM SUPERVISOR (Sign with full name) (Sign with full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- -- SUPERVISOR’S EVALUATION SHEET Student name: Phạm Hồng Đăng Student ID: 20151091 Class: 20151CLA2 Student name: Phùng Nguyễn Việt Hùng Student ID: 20151281 Class: 20151CLA4 Major: Automation and Control Engineering Technology Project title: Automation and Control Engineering Technology Supervisor: Associate Ph.D Ngô Văn Thuyên EVALUATION 1. Content of the project:. Approval for oral defense? (Approved or denied).
(in words: ………………………………………………… ) Ho Chi Minh City, June 25th, 2024 SUPERVISOR (Sign with full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- PRE-DEFENSE EVALUATION SHEET Student name: Phạm Hồng Đăng Student ID: 20151091 Class: 20151CLA2 Student name: Phùng Nguyễn Việt Hùng Student ID: 20151281 Class: 20151CLA4 Major: Automation and Control Engineering Technology Project title: Automation and Control Engineering Technology Supervisor: Associate Ph.D Ngô Văn Thuyên Name of Examiner:. Content and workload of the project. Approval for oral defense? (Approved or denied). Marks: (In words: .) Ho Chi Minh City, June 25th, 2024 EXAMINER (Sign with full name) ACKNOWLEDGEMENT A graduation thesis is a student's scientific research project, a necessary requirement for every student before graduation.
Researching a new problem for students is a difficult and confusing task. The guidance and help of teachers were crucial in completing our projects. First of all, we would like to thank the teachers of Ho Chi Minh City University of Technology and Education, especially the lecturers in the Control and Automation Engineering Department of the International Training Department for their dedication and dedication. wholeheartedly imparted knowledge to us during the 4 years we study at school.
In particular, we would like to send our sincere thanks to Associate Professor, PhD. Ngo Van Thuyen for wholeheartedly helping, creating conditions and closely following us during the process of implementing and completing this thesis. We would like to express our sincere gratitude to Mr. Nguyen Tan Thuong, Meng.
Luong Cong Trinh, Mr. Vu Tuan Son, and the staff of Pro Storage Solutions Technical Company Limited for their assistance in helping us complete the thesis. Finally, we would like to thank the reviewers and the teachers in the thesis defense committee for taking the time to comment and help us during the thesis defense process. Wishing all teachers good health and success in research and teaching.
i LIST OF ACRONYMS PLC Programmable Logic Controller WIFI Wireless Fidelity DO Digital output IO Input/output DC Direct current UI User Interface OPC DA Open Platform Communications Data Access OPC UA Open Platform Communications Unified Architecture OCR Optical Character Recognition HMI Human Machine Interface ROI Region of Interest ii ABSTRACT In this project, we developed software to operate the labelling station for washing machines. The system is currently in operation at the Prosst’s client and has significantly improved the factory's productivity while reducing the resources needed to operate the system. Additionally, our team designed a simulation model to provide a comprehensive overview of the station's operations, optimized the defective label classification problem, determined label placement coordinates on washing machines for the robot, designed a monitoring system, and implemented an OPC communication protocol to connect various devices. The main components of the system include a programmable logic controller (PLC) that serves as the central processor, receiving processed results from the vision program and sending control commands to a robot, a system virtual station operating in RobotStudio software, a vision processing program and a web server for monitoring the program's results and an HMI for system operation and monitoring.
Upon completion, the system demonstrated stable connectivity between software and devices, accurately classified label samples, identified label placement coordinates with minimal error, and featured an intuitive monitoring interface for ease of operation. iii APPENDIX ACKNOWLEDGEMENT .i LIST OF ACRONYMS .iv APPENDIX IMAGE .ix APPENDIX TABLE. xiii CHAPTER 1: OVERVIEW .4 CHAPTER 2: THEORETICAL BASIS. The automatic labelling control and monitoring system.
Introduction of the system. Programmable Logic Controller (PLC). OPC and TCP/IP communication. Introduction to OPC.
Classify two popular OPC protocols. Introduction to Kepware (KepserverEx Configuration 6). Computer Vision in Automatic Process. Optical Character Recognition (OCR).
The advantages of OCR. You Only Look Once (YOLOv8). Introduction to YOLOv8. Operation of YOLOv8.
Introduction to OpenCV library. The application of Opencv. Calibration camera and calculate real word coordinate. Camera calibration process.
Calculate the real-world coordinates .1 Introduction to OpenVINO. Operation of OpenVINO. Web Server with Node.js and Flask .24 CHAPTER 3: HARDWARE DESIGN. Hardware design requirements.
General diagram of the system and block diagram. Vision devices selection. OCR reading position. Location to calculate label coordinates.
Vision Light LED. Camera basler Aca2500-14gc. I/O devices and PLC selection. PLC and expansion module.
Step Motor and Gearbox. Wiring diagram between Qy40p, Arduino, TB6600, Relay and Nema17. Aptomat Chint MCB 2 phases 32A 6kA. Operational monitoring and display equipment and module Wi-Fi.
The light tower. HMI Delta DOP-107 DV. Basic specifications of ROBOT ABB IRB 1200-5/0.44 CHAPTER 4: SOFTWARE DESIGN .1 System software requirements .2 Building robot stations and programs on RobotStudio software. Design of system operating stations on RobotStudio.
Labelling tool and washing machine model. Created Signals in Robot Controller. Basic Movements and Move Instruction. System operation flowchart.
Create a washing machine model on RobotStudio software. Conditions for washing machine model moving on RobotStudio. Robot pasting label to the model. Program Distribution in Rapid.
PLC, RobotStudio, Vision system and Kepserver Communication. Setup connection between PLC and Kepserver. Communication between RobotStudio and the Kepserverex software. Communication between the Vision program and the Kepserverex software.
Program for the label sorting process. Program to find the labeled area. Alarm designing and History displaying. Right access designing.
Webserver monitoring vision processing.78 CHAPTER 5: EXPERIMENTAL RESULT. Detecting Label Error. Detecting Good Label. HMI during operating process.
Running result on RobotStudio. Video of the system results. 91 viii APPENDIX IMAGE Figure 1. The average share of investment in automation (%).
Pasting label possibility. The figure of washing machine. Collect sample images to calculate the intrinsic matrix. Chessboard corners are detected.
Formats and hardware that OpenVINO supports. The operation of OpenVINO. Diagram of the operation of the web server. General diagram of the system.
Hardware setup for reading OCR. Set up hardware for finding labelling positon. VSTECH VL-DR SERIES LED. Camera basler Aca2500-14gc.
Qx40 Wiring diagram. Qy40p wiring diagram. Wiring diagram between Q03 UDVCPU, Qx40, Qy40p and IO devices. Wiring diagram between Qy40p, Arduino Mega 2560, TB6600 and Nema17.
MCB 2 phases 32A 6kA. Delta DOP-107 DV. PLC, HMI and Wi-Fi module wiring diagram. The working area of robot ABB IRB 1200-5/0.
Communication and Industrial Networks of RobotStudio. Setting coordinates for the tool. Washing machine 3D model. Cylinder Pushing signal .Operation flowchart of the system.
Create WM model in Smart Components. Robot, Conveyor and PlaneSensor. Signals from the controller to control model move/stop. PLC IP address and ports configuration.
Kepserver Configuration for connecting with PLC. Tags and Registers of PLC in Kepserver. Create a Client for controlling RobotStudio on KepserverEx. OPC DA RobotStudio Client data.
Enabling the OPC UA server. Getting the address of nodes client. The label Sample. Flowchart of OCR function.
The result when the image is tilted. The result when the image is deskew. Size between the YOLOv8 model and the OpenVINO model. Inference time between OpenVINO and YOLOv8.
Crop OCR part. Contours are found. Detection results row 1. Detection results row 2.
The HSV Mask. Flowchart of Find labelling area function. The result of the detected coordinate. Flowchart of the main program.
X and Y values in Graph and history table data. The not-good label. The light tower indicates a red light. The NG result.
The good label. The light tower indicates a green light. The Good result. HMI output data displayed during the operation.
HMI errors and warnings. Robot in stopping state during labelling process. Video about the system operation .88 xii APPENDIX TABLE Table 2. Some configuration options of pytesseract.
The options page segmentation mode. Some main parameters of the camera. Qx40 main technical specifications. Qy40p technical specifications.
E3F-DS30C4 Technical Specifications. Omron S8VK-C24024 Technical specifications. Robot Working Range. Rapid Programs distribution.
PLC programs description. Overview CHAPTER 1: OVERVIEW This chapter provides an introduction to the thesis, defines the contents, and project scopes, points out the research questions dealt with, and describes the layout of this thesis. Introduction Nowadays, industrial automation is rapidly growing and essential. Businesses are investing in automation, robotics, and technology to boost output and cut costs.
The 2022 McKinsey Global Industrial Robotics survey projects that automatic systems will comprise 25% of capital expenditures over the next five years which is shown in Figure 1. With increasing numbers of manufacturing enterprises, competition is intense, making quality a necessary requirement rather than a mere competitive advantage. Enhancing factory machinery to improve productivity, and efficiency, and reduce labor costs is crucial. The average share of investment in automation (%).
On a typical workday, manual labelling of thousands of washing machines with various models and positions requires focused attention from workers to avoid label printing errors that can lead to customer dissatisfaction. Implementing a labelling system reduces errors, saves time, and enhances productivity and efficiency compared to manual methods.