VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY ADVANCED PROGRAM IN INFORMATION SYSTEMS NGUYEN MINH TU - DO THANH XUAN AN ANNOTATION TOOL FOR MACHINE LEARNING BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS Ho Chi Minh City, 2021 NGUYEN MINH TU - 16521839 DO THANH XUAN - 16521479 AN ANNOTATION TOOL FOR MACHINE LEARNING BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS THESIS ADVISOR DR. DO TRONG HOP Ho Chi Minh City, 2021 ASEMNTCOIE ACKNOWLEDGMENTS First of all, the authors want to say thank you to the University of Information Technology and all of the lecturers in Information System department for tutoring and teaching important knowledge and treasure not only in university, but also in real life through 4 years so that the authors can qualified to complete this thesis. The authors want to send their best regards and extend their thanks to Mr. Do Trong Hop and Mr.
Nguyen Thanh Binh for directly giving advice, corrections, helping and supporting the authors through the time implement and working on the thesis. All these encouragement, feedback and inputs are treasure motivation when the author struggling for research and implement this project. The authors also extend their thanks to Mr. Ngo Duc Thanh for giving inputs and advices so the report and thesis can complete 100%.
The final words, the authors want to say thank you all to family and friends for those encouragement through all the research project. However, due to the authors’ knowledge and experience is still limited so that mistakes and shortcommings are inevitable. Hence, the author sincerely looking forward to receiving helpful contribution, advices and feedbacks from the lecturers to complete, fluent and be prerequisite for me to be able to implement other projects in the future. Again, thank you for all your help.
Thank you sincerely. UNIVERSITY OF INFORMATION Advanced TECHNOLOGY Education Program ADVANCED PROGRAM IN INFORMATION SYSTEMS THESIS PROPOSAL THESIS TITLE: OBJECT DETECTION Advisor: Do Trong Hop Duration: (From 14" Aug to 11" Jan , 2021) Students: Nguyen Minh Tu - 16521839 Do Thanh Xuan — 16521478 Contents: (Describe the details of what to be done, scope, objectives, methodologies, expected results) 1. Introduction: Nowadays, with the rapid growth of AI and machine learning. For the algorithm to perform efficiently, it needs a learning source and a large dataset.
Managing and creating datasets is a very time consuming job. Thus, it requires the effective provisions of tools and secondary algorithm. Scope: Main functionalities: ° Manage Videos, Images, Training dataset. ° Labeling observation every single picture, video, frame.
* Export types of dataset from the labelled sources. Objectives: - Learn how the dataset works and the structure of the common algorithm platforms. - Creating an Object Detection dataset using Yolo and Tensorflow to manage, generate datasets, training set and test set. Methodologies: 3 main steps Programing language: C#, Python s Survey main machine learning framework: 1.
Investigate, Training, Testing and Running provided dataset (COCO, TINY). Analyze sample dataset. ¢ Developed an application support manage dataset, labeling objects and export training dataset compatible with analyze platforms s Using the result exported to verify data which is images, videos. Research time lines: (Plan of action: describe the planning and the assigned tasks for each students) Task Start Date | End Date | Assigned to Meet advisor Mr.Binh to contact and have 01/08/2020 Tu, Xuan idea about project Research about technology and framework | 03/08/2020 | 09/08/2020 | Tu, Xuan for working Survey main machine learning framework: 10/08/2020 | 16/08/2020 Tu Investigate, Training, Testing and Running provided dataset(COCO, TINY).
Analyse sample dataset. 17/08/2020 | 15/09/2020 Xuan Develope an application which support 17/08/2020 | 25/11/2020 Tu manage images, videos, labeling objects and export training dataset compatible with analysed platforms Using the result exported to verify data 25/11/2020 | 02/12/2020 | Tu, Xuan which is image, video. Testing and completing the application 02/12/2020 | 9/12/2020 | Tu, Xuan Document writing and Reporting 09/12/2020 | 15/12/2020 | Tu, Xuan Approved by the advisor(s) Ho Chi Minh city, 18/9/2020 Signature(s) of advisor(s) Signature(s) of student(s) TABLE OF CONTENT calle ASSESSMENT COMMITEE. TH 000000000406 850 iii TABLE OF CONTTIEÌNTT.
00000000 00)vi LIST OF EIGUIES. << << 5£ <5 4 HH 00000010 00ix LIST OF TABLES LIST OF ACRONYMS. xiii ABSTRACT OPENING. 5< HH TH HH TH nh T00 n3 0741.0340401 XV CHAPTER I: PROJECT OVERVIEW .1 Problems and questions: .1 Problem and statement:.
Annotation Tools Research: 1. CHAPTER 2: THEORETICAL BASI 2.- 5s nung HH HH niệu 8 2.2 Object Detection Platforms and Algorithms: .1 Yolo’s framework: DARKNET.5 Set up environment on Darknet/yolo V4: .6 Pros and Cons:.2 Tensorflow/SSD_resnet:.,nH HH HH HH He Ha 23 2.4 Set up environment on Tensorflow/SSD_resnefS:.5 Pros and Cons:.-- 55 nu ngư ướt CHAPTER 3: GROUNDTRUTH SYSTEM DESIGN AND ANALYSIS.2 Groundtruth Main Functions:.2 Non-Function Requirements: 3.3 Use Case Diagrams.1 Use case diagram lisf:.2 Use case diagram desCrÏDfÏOH:.1 Manage dataset use case diagraI:.2 Manage data source use Case dỉagram:.3 Manage labels use case dỉag8raIm:.4 Export dataset use case diaỹraIm:.1 Add dataset sequence đỉaðTaI:.2 Delete dataset/data sourc sequence diagrams. Add data source sequence đỉagram:. Delete data source sequence diagram:.
Edit data source sequence diagram: 3. CHAPTER 4: TESTING ON YOLO AND SSD_RESNET. Deploy Groundtruth For Generating Dataset For Yolo An Tensorfow: .2 Yolo training and testing deploymen(:.1 Trainings oe ccceecssseccssseecessvecessevecesseeeessnsccssnessssvecssnneccsnnecesnueecssneceesnneceunneeesneeess 44 4.3 SSD_Resnet Training And Testing Deployment:.1 Trainings occ ccceeccsssecessseecessvecessevecesnsesssnnsecssseceesnnecessnecesnneessneeeessneceesneceunneseeneeets 50 vii 4. on HH HH HH HH HH ghi 52 4.3 Auto Image Annotation for Tensorflow:.
CHAPTER 5: CONCLUSION AND FUTURE DEVELOPMENT:. Advantages Of Groundtruth:. Disadvantages Of Groundtruth: 5. Improvement/Upgrade In The Future:.ccsscsssssesssssessscsscsncssccsecsncsnsencencenecsncsscenceneesecancsucencesceneeaecanceneenceneeneeseeses 59 viii LIST OF FIGURES œEElt› Figure 1-1: User Interface of labellmg application.-----¿-5-5- 55+ ssss+s+xsescee2 Figure 1-2: Graphic User Interface of CV ATT.
+ xxx seerrrrrrereerkrkree 3 Figure 2-1: Example of Image Annotation. Figure 2-2: A Manual Annotation Tool For Machine Learning Figure 2-3: Auto Image Annotation Process. Figure 2-4: Bounding Box Annotation Feature Figure 2-5: Cuboid Box Annotation Feature.------ 5c cccc+xersrereersrrerrere 1 Figure 2-6: Line Annotation Feature.----s-c:ccccxcccctettrterrerrrrerrrrrrrerrrrrrrree 2 Figure 2-7: Darknet Framework LLOBO.--- - - + 6 SE Sk*k+kEEEEkEEkekekrkkrkrkrkeree 3 Figure 2-8: Performance Of Yolov4 Compared With Other Object Detection Algorithms. Figure 2-9: Coco Dataset’s Performance On Different Algorithms Figure 2-10: Yolo Dataset.cececcecsssesessseessesseeesesesescseeceeseeeeseseseaeeceeeseseeeseaeenseeeeeasees 5 Figure 2-11: - Coordinate Text File Explanation.
----‹-+-++c+c+scs+szx+cs> 6 Figure 2-12: - Folder Obj of Yolo Dataset.cccececesseeeeseseeeeseseseseeeesesesesssneseaeseees 7 Figure 2-13: - File Obj.Data Của Yolo Dataset.-- 5c cccccsterererrrrrer 7 Figure 2-14: Obj.Names Of Yolo Dataset. c-cc St tr ưec 8 Figure 2-15: Obj.Names Of Yolo Dataset. cece eessseeseseeseseseseeeseeseseseeeeneseaeseeee 8 Figure 2-16: Alexeyab’s Darknet Github. cceceseseeesesseseseseeeneneeeeeeseseseaeeneeeeeeeeees 9 Figure 2-17: Download And Install CUDA Toolkit.
eee ceeeeeeseeneneseee 9 Figure 2-18: Download And Install OpenCV Figure 2-19: Cmake Configuration With CUDA Toolkit. Figure 2-20: Configuration File And Pre-Trained Model. Figure 2-21: Config Cmake To Unzip The Environment From Alexeyab. Figure 2-22: Building Darknet/Yolo Environiment.
--- c5 +++c+c+scxcrexexee 2 Figure 2-23: Darknet/Yolo EnVITOIeIII. ó6 + tk 22 Figure 2-24: Tensorflow Library LOBO. - ¿+ SE Sk*k‡EEEEEEEkekeErkkrkrkrkrtee 24 Figure 2-25: VOC Dataset For TensOr[ÏOW. óc tt St Sky 24 Figure 2-26: Tensorflow metadata files.
¿S5 S SE 11111 re 25 Figure 2-27: Data from XML Ẩile.25 Figure 2-28: Indicators Files For Environment To Training.------ ----26 ix Figure 2-29: Data Sources Formatted With MDỐ. 6-65 cccetssrerrrererereek 26 Figure 2-30: Labels From Tensorflow DatasSet. 6 Sàn 27 Figure 2-31: Annotation File For Tensorflow. 6-5 csseeveeerrrrerererexee 27 Figure 2-32: Data Stored In Annotated File.
cccesseeceseseeeneeeeeeseseeeseneeeneeeeeseeees 28 Figure 2-33: Label_Map.- «5S rên 28 Figure 2-34: Data Source Directory Stored In Train.------- - -29 Figure 2-35: Data Source Directory Stored In Vai. cece -----ccccccscs> 29 Figure 2-36: Checking GPU Support. Figure 2-37: Install Python. Figure 2-38: Installing Virtualenv.
Figure 2-39: Import Tensorflow Library. Figure 3-1: Groundtruth user interface.cceeeeseseseeeeseceeeseseeceeeeeeeseseseeeseeneneeeeeeaes 33 Figure 3-2: Main Use Case điagraim.- «c5 St TH ưếc34 Figure 3-3: Manage Dataset Use Case Diagram.ccccceceesseseeeeeseeteseseseensneeaeseees 35 Figure 3-4: Manage Datasource Use Case Diagram.cccecscscsseesesteteseseseeneseneeee 36 Figure 3-5: Manage Labels Use Case Diagram.cccccsccseseesenseseseeteesesesneneseneees 37 Figure 3-6: Export Dataset Use Case Diagram.--¿- 5S se 2ccxcxsrererrrreree 38 Figure 3-7: Add Dataset Sequence IDiagram. ¿6555 S+cx+xsrtrerrrkerer 39 Figure 3-8: Delete Dataset/Data Source Sequence Diagram.------‹---- 39 Figure 3-9: Add Data Source Sequence Diagram. Figure 3-10: Delete Data Source Sequence Diagram.
Figure 3-12: Groundtruth Database.cccccccccesesseseseseseseeseseseeseneseseseeeeensseseseeeeeseseseenesenseee 41 Figure 3-14: Setdb. c2 St SH eưườ42 Figure 3-15: Mediasource TFOÏ€T.- «ch HH 42 Figure 3-16: Groundtruth FFOÏ€T.-- - «xxx kg re43 Figure 4-2: Set Up Dataset into EnvirOnIeIIL. 5 Server 44 Figure 4-3: Input Commands For Training OÌO.---¿- ¿555 ++s+s*cvxeeeeexexev 45 Figure 4-4: Training Process Of O]O. càng Hư 45 Figure 4-5: Indicators To Notice When Training, .ccccceceseseseseeeeseseseeneeeneneeeeeeees 46 Figure 4-6: Training Ñ €SuÌ(.
- cà TT HH HT 46 Figure 4-7: Configuration For Yolov4-Obj.Cfg Figure 4-8: Input commands for testing. Figure 4-9: Machine Detecting Object. Figure 4-10: YOLO detection Result Number Í. 5c c5 5csstsvrvrerererexev 48 Figure 4-11: YOLO detection Result Number 2.
5c 555 tt svrvrerererexev 49 Figure 4-12: Json EÏÏe. ch SH HH HH HH HH49 Figure 4-13: Active Environment For Tensorflow And Create “Train.Record” And “Val. Figure 4-14: Complete Dataset Of Tensorflow. Figure 4-15: Configuration For Training.cccccecesesesessesescseesesesesessesesesenesenseeeseeeaes 51 Figure 4-16: Input Command For Training For SSD_Resnet On Tensorflow.
51 Figure 4-17: Training Process.ccccccsesesssssseseseseseseseesensseseseseeeesseseseeeaessseseeneaeseees 52 Figure 4-18: Export Saved Model Environment Commands.---- ‹-‹-+ 52 Figure 4-19: Export Saved Model Complete.ccccsssessseeseseeeeessesesesesesenseeeeseeees 53 Figure 4-20: Command Line for Validating Detection On Tensorflow/SSD_Resnet.53 Figure 4-21: Tensorflow detection Result Number Ï. - - 5s +<sc+xexses+ 53 Figure 4-22: Tensorflow detection Result Number 2. 555 5<<<sc<c-x-x-x/ Figure 4-23: Result Of Validating Data. Figure 4-24: Data Source’s Metadata As An Xml File.
Figure apendix 1: YOLO Test Result For Standard Test. Figure apendix 2: SSD_resnet Standard Test For SSD_Resnet Result. 65 Figure apendix 3: YOLO Test Result For Contrast Boosted Test.-- -‹ 65 Figure apendix 4: SSD_resnet Contrast Boosted Test For SSD_ Resnet. 66 Figure apendix 5: YOLO Result Of Low-Resolution 'Test.
5s 5+ secsxsc++ 66 Figure apendix 6: YOLO Low Resolution Test ResuÏt.--- 5555 <<<67 xi LIST OF TABLES calle Table 3-1: Use case diagram list. e cece eee ee +.) xii LIST OF ACRONYMS CUDA: Compute Unified Device Architecture GPU: Graphics processing unit CC: Compute Capability GCC: GNU Compiler Collection MSVC: Microsoft Visual C++ CuDNN: Deep Neural Network library CVAT: Computer Vision Annotation Tool YOLO: You Only Look Once xiii ABSTRACT Thesis project “An image annotation tool” is a project focus on users who demand a tool which generate and manage datasets for training machine learning in a fast, convenient and accuracy way. This project is to build and developed an annotation tool which helps to create, generate and manage datasets response to the demand of basic functions such as annotating and labeling object, managing datasets and data sources (pictures and video) of those datasets in a convenient way for the users. After a brief about this project, the authors had planned to implement project as below: ⁄ Research of steps how to detect an object.
v Research about Darknet/YOLO ⁄ Research about Tensorflow/SSD resnets Y Developed and build up Groundtruth application to labeling, annotating and managing dataset and data source. Y Set up environments for Darknet/YOLO and Tensorflow/SSD_resnets. Vv Using datasets generated from GroundTruth application to training and tesing on both environment and algorithms. xiv OPENING With development of AI and machine learning all over the globe, especially object detection because of the high demands, access, exploit and need of using them in real life has been larger than ever.
Especially in Big-Tech companies. Object detection’s scope applies not only to individual users/small-scale businesses but also big corporation. Following with small-scale businesses and that is one of the hardest tasks is to create their own annotation tool. There are two popular kind of annotation tools, one of which is powerful but they’re not open-source, the other one is free but lacks many functionalities.
Our software provides a free of charge annotation tool, better features than some nowadays tools. Our targeted users are: > Individual user. > Small-scale business company.