VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY NGUYỄN THANH HOÀNG HẢO THIẾT KẾ BỘ PHÁT HIỆN LỖI BỀ MẶT CỦA ĐẾ CẮM CPU BẰNG HỌC MÁY DESIGN OF SOCKET’S SURFACE DEFECT DETECTION USING MACHINE LEARNING Major: Mechatronics Engineering Major code: 8520114 MASTER’S THESIS HO CHI MINH, June 2024 THIS THESIS IS COMPLETED AT HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY – VNU-HCM Supervisor: Dr. Tran Viet Hong (Please fill in the supervisor’ full name, academic rank and signature) Examiner 1: Dr. Le Thanh Hai (Please fill in the examiner’ full name, academic rank and signature) Examiner 2: Assoc. Le My Ha (Please fill in the examiner’ full name, academic rank and signature) This master’s thesis is defended at HCM City University of Technology, VNU- HCM City on 15th June, 2024.
Master’s Thesis Committee: (Please write down full name and academic rank of each member of the Master’s Thesis Committee) 1. Nguyen Quoc Chi (Committee chair) 2. Nguyen Duy Anh (Secretary) 3. Le Thanh Hai (Examiner 1) 4.
Le My Ha (Examiner 2) 5. Ngo Ha Quang Thinh (Committee member) Approval of the Chair of Master’s Thesis Committee and Dean of Faculty of Mechanical Engineering after the thesis being corrected (If any). CHAIR OF THESIS COMMITTEE DEAN OF FACULTY VIETNAM NATIONAL UNIVERSITY - HO CHI MINH CITY HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY THE TASK SHEET OF MASTER’S THESIS Full name: NGUYỄN THANH HOÀNG HẢO Student ID: 2170095 Date of birth: 12 November 1996 Place of birth: Binh Dinh Major: Mechatronics Engineering Major ID: 8520114 I. THESIS TITLE (In Vietnamese): “Thiết kế bộ phát hiện lỗi bề mặt của đế cắm CPU bằng học máy” II.
THESIS TITLE (In English): “Design of socket’s surface defect detection using machine learning” III. TASKS AND CONTENTS: Research introduction and theory to solve the problem of detecting surface defects of sockets. Through experiment, build a machine learning model and evaluate the performance when solving problems. THESIS START DAY: (According to the decision on assignment of Master’s thesis): 4th September 2023 V.
THESIS COMPLETION DAY: (According to the decision on assignment of Master’s thesis) 10th October 2024 VI. Tran Viet Hong Ho Chi Minh City, date 25th May 2024 SUPERVISOR HEAD OF DEPARTMENT (Full name and signature) (Full name and signature) DEAN OF FACULTY OF MECHANICAL ENGINEERING (Full name and signature) Note: Student must pin this task sheet as the first page of the Master’s Thesis i ACKNOWLEDGEMENTS I would like to express my sincere gratitude to Dr. Tran Viet Hong for his wholehearted encouragement, guidance, and support, which facilitated my completion of this thesis. I also wish to convey my deep appreciation to the esteemed professors of Ho Chi Minh City University of Technology for imparting valuable knowledge and experience that enabled me to complete my training program.
Thank you to the Postgraduate Department for their enthusiastic assistance throughout my study and thesis writing process. Lastly, I would like to thank my friends, family, and colleagues for their encouragement, empathy, and help during this time. Ho Chi Minh City, date 25th May 2024 Student Nguyễn Thanh Hoàng Hảo ii ABSTRACT In computer hardware, processor socket is designed to provide mechanical and electrical connection between microprocessor and printed circuit board (PCB). In microprocessor industry, sockets are widely used during electrical and functional testing of processor (CPU).
Without them, PCB cannot contact properly with processor for power and electrical signal transmission. Due to its important application in any processor facility, sockets need regular maintenance and repairing to ensure electrical their electrical characteristics. However, it is very challenging to detect all the surface defects by manual inspection with microprocessor in manufacturing line. In practice, sockets are put under microscope and scanned by technicians, then detectability and efficiency depend heavily on each person’s assessment and skill.
Therefore, defected sockets cannot be captured in repairing station and caused bad performance or even electrical failure or damages to processor during production. Moreover, manual inspection is not time-efficient and always varies on person-to-person ability (eye vision, awareness, agility). To overcome this problem and get rid of human mistakes, it is essential to automate socket’s defect detection. This study presents novel algorithm for surface defect detection in processor socket based on machine learning (ML).
The hardest obstacle in this field of study is demand for huge quantity data to train the network to recognize the different defects might exist on sockets. Insufficient data collection will result in inaccurate results or underfitting problems. Furthermore, various types of noise factors induce during visual inspection, such as lighting, vibration, camera focal length, etc. In this regard, ML, which is a distinct method that shows high accuracy and effective technique for detecting defect, which is easy to understand and deploy for using.
The proposed model can successfully detect various socket defects to help prevent cost of microprocessor defect in production floor. iii The study and demonstrated results revealed that ML or deep learning (DL)’s performance in surface defect detection. It exhibited high detection accuracy, with over 95% and quick detection time. The developed methodology provides strong evidence to encourage ongoing and extensive research in same field of study.
The study’s findings indicate that using the latest optimizer tool kit can enhance model performance in terms of accuracy with less computational resource. This shows potential for faster convergence time and cross platform deployments from server to mobile device. While existing studies have typically focused on other fields of such as fruits and vegetables classifications, road crack detection, defect turbine, damaged fishing net, this study pioneered the methodology of DL for socket defect detection while considering influence of noise factors. Consequently, this research can be a valuable reference for other researchers to select suitable model and optimizer method for their application in field of surface defect detection.
This thesis consists of five separate chapters. Chapter 1 presents background and the reason for choosing the topic, scientific and practical benefit of the research topic. Chapter 2 analyze and evaluate existing research works that related to the topic and study objectives. In this section, we study various researches, theories and methodologies and raise existing problems that the project needs to focus on researching and solving.
Consequently, chapter 3 cited theory and technique used in this research. This section describes method, technique and data utilized to train models and the way optimize, enhance model performance. Brief description of the job conducted in this research is placed in chapter 4. Then, there’s discussion based on results obtained during the research of the topic and comparison with the research results of other authors through reference documents.
Lastly, chapter 5 shows conclusion and recommendation for further research. iv THE COMMITMENT OF THE THESIS’ AUTHOR I hereby declare that this is my own research work and has been conducted under the scientific guidance of Dr. Tran Viet Hong. The research content and results presented in this thesis are truthful and have been carried out by the author without violating any research ethics.
The data in the tables and figures used for analysis, comments, and evaluations were collected by the author from various sources, which are clearly cited in the reference section. Additionally, this thesis includes some comments, evaluations, and data from other authors and organizations, all of which are properly quoted and referenced. Should there be any detection of fraud, I take full responsibility for the content of my thesis. Ho Chi Minh City University of Technology is not responsible for any copyright or intellectual property violations that I may have caused during my work (if any).
Nguyễn Thanh Hoàng Hảo v CONTENTS ACKNOWLEDGEMENTS. ii THE COMMITMENT OF THE THESIS’ AUTHOR .v LIST OF FIGURES. vii LIST OF TABLES. Deep learning method.
MATERIALS AND METHODS. Image acquisition and processing. ROI segmentation and feature extraction. Convolutional neural network and transfer learning.
Artificial Neural network. Convolutional neural network (CNN). Optimization with OpenVINO method. RESULTS AND DISCUSSION.
Data augmentation and balancing. The performance of model against different noise .72 vii LIST OF FIGURES Figure 1.1 CPU socket in personal computer (PC) [1] .2 Semiconductor test socket [2] .3 Different types of socket defects: Contaminated socket with foreign material (a), burnt socket (b), hard debris on socket pin (c), stained socket surface (d), bent pin (e), bridging pins with metal particle (f) [3] .4 Illustration of manual inspection with microscope [4] .1 Composition of visual inspection system .2 CCD image sensor [37] .3 CMOS image sensor [38] .4 EVOCAM II model .5 Example of CPU socket .6 Classification of image denoising.7 Example of Gaussian noise applied on socket image .8 Example of Uniform noise on socket image .9 Example of impulse noise (salt and pepper noise) on socket image .10 Histogram equalization application .11 Edge detection with Canny .12 ROI extraction by Hough transform .13 Sketch of an artificial neuron [39] .14 Artificial Neural network .15 An illustration of the position of deep learning (DL) within the area of artificial intelligence (AI) [40] .16 Schematic diagram of convolution operation [41] .18 Pooling layer operation [42] .19 Fully connected layer [43] .24 Performance of top accuracy model [36] .1 The experiment setup .2 Image from system setup .3 Example of sample.4 Collection of input images .6 Augmentation method example, Original image (a); Rotated image (b); Random cropped image (c); Flipped image (d) .7 Validation accuracy comparison between subset .8 Load pre-trained model and optimizer .9 Run classification model .10 The accuracy of training process .11 Classification result example .12 Training and validation loss .13 Training and validation accuracy .14 The true recognition with bad label .15 Fail recognition with bad label .16 Improvement of accuracy after optimization with OpenVINO .17 Example of a crack image polluted with different noise sources .18 Performance of enhanced model against different noise sources .19 Classification of proposed model with background noise .63 ix LIST OF TABLES Table 3.1 EVOCAM II camera specification .2 Summary of the study dataset .3 Typical CNN architecture .1 Effect of Median filter against multiple noise .2 Effect of Gaussian filter against multiple noise .3 Process step for defect detection system .4 Comparison of evaluation metrics among models .5 Prediction result with unseen images .7 Performance of the proposed model against various noise sources. INTRODUCTION Processor socket plays an important role for computing systems to provide mechanical connection, power transition and data communication between processor (CPU) and printed circuit board (PCB). In semiconductor industry, sockets are widely used during electrical and functional testing of processor.
Without them, PCB cannot contact properly with CPU for power and electrical signal transmission. Due to its important application in any processor facility, sockets need regular maintenance and repairing to ensure electrical their electrical characteristics. With its versatile fundamental in integrated validation system, the demand for high-quality sockets has been insisting for years in semiconductor industry. The electric conductivity and data integrity of socket are essential in product yield and overall manufacturing productivity.1 CPU socket in personal computer (PC) [1] Figure 1.2 Semiconductor test socket [2] 2 However, the most challenge faced during production in any factory facility is surface defect induced by various factors as environment, human or process.
There are variety of socket defects as contamination with foreign material; dent or burnt pin; damaged pad; scratched or stained surface; imprinting of socket by hard debris; bent pin; bridging contact pins with metal particles; etc. Illustration of these mentioned defect can be shown in figure 1. Socket damages can result in poor electrical connectivity, reduced validation capacity, invalid testing failure that resulted in significant drop in yield. Moreover, sensitive defects as conductive metal particles can contribute to short circuit, causing widespread and severe damages to processors.
Early detection and removal of defect factors from socket become critical during maintenance to maintain socket quality before releasing to production line.