NATIONAL UNIVERSITY OF HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY FACULTY OF COMPUTER SCIENCE AND ENGINEERING OOO BK TP.HCM GRADUATION THESIS DEVELOPMENT OF A MOBILE APPLICAIONT TO PROCESS HOUSE OWNERSHIP CERTIFICATE FIELD OF STUDY: COMPUTER SCIENCE COUNCIL: CLC_KHMT 2 INSTRUCTOR:ASSOC. QUẢN THÀNH THƠ REVIEWER: ASSOC. BÙI HOÀI THẮNG —O0O— STUDENT: LÊ BÁ THÀNH 1552340 Ho Chi Minh city, July 2021 ĐẠI HỌC QUỐC GIA TP.HCM CỘNG HÒA XÃ HỘI CHỦ NGHĨA VIỆT NAM ---------- Độc lập - Tự do - Hạnh phúc TRƯỜNG ĐẠI HỌC BÁCH KHOA KHOA:KH & KT Máy tính NHIỆM VỤ LUẬN ÁN TỐT NGHIỆP BỘ MÔN:KHMT Chú ý: Sinh viên phải dán tờ này vào trang nhất của bản thuyết trình HỌ VÀ TÊN: Lê Bá Thành MSSV: 1552340 NGÀNH: KHMT LỚP: CC16KHM1 1. Đầu đề luận án: Development of a mobile application to process house ownership certificate 2.
Nhiệm vụ (yêu cầu về nội dung và số liệu ban đầu): ✔ Design and implement a UI for friendly users to easily interact and retrieve information. ✔ Define which part of the certificate needs to extract useful information. ✔ Build a pipeline to process the right part of the certificate to retrieve the location of the land on the maps for quick review. ✔ Improve the performance of the processing location for more faster.
✔ Evaluating the whole process and the accuracy of application. Ngày giao nhiệm vụ luận án: 4. Ngày hoàn thành nhiệm vụ: 5. Họ tên giảng viên hướng dẫn: Phần hướng dẫn: 1) Quản Thành Thơ 2) 3) Nội dung và yêu cầu LVTN đã được thông qua Bộ môn.
CHỦ NHIỆM BỘ MÔN GIẢNG VIÊN HƯỚNG DẪN CHÍNH (Ký và ghi rõ họ tên) (Ký và ghi rõ họ tên) PGS. Quản Thành Thơ PHẦN DÀNH CHO KHOA, BỘ MÔN: Người duyệt (chấm sơ bộ): Đơn vị: Ngày bảo vệ: Điểm tổng kết: Nơi lưu trữ luận án: TRƯỜNG ĐẠI HỌC BÁCH KHOA CỘNG HÒA XÃ HỘI CHỦ NGHĨA VIỆT NAM KHOA KH & KT MÁY TÍNH Độc lập - Tự do - Hạnh phúc ---------------------------- Ngày tháng năm PHIẾU CHẤM BẢO VỆ LVTN (Dành cho người hướng dẫn/phản biện) 1. Họ và tên SV: Lê Bá Thành MSSV: 1552340 Ngành (chuyên ngành): KHMT 2. Đề tài: Development of a mobile application to process house ownership certificate 3.
Họ tên người hướng dẫn/phản biện: PGS. Quản Thành Thơ 4. Tổng quát về bản thuyết minh: Số trang: Số chương: Số bảng số liệu Số hình vẽ: Số tài liệu tham khảo: Phần mềm tính toán: Hiện vật (sản phẩm) 5. Tổng quát về các bản vẽ: - Số bản vẽ: Bản A1: Bản A2: Khổ khác: - Số bản vẽ vẽ tay Số bản vẽ trên máy tính: 6.
Những ưu điểm chính của LVTN: The thesis addresses some challenging issues when processing an image of a house ownership certificate, which is text extraction and especially identifying the location of the land. The student conducted a literature review, proposed suitable solutions, which are based on state-of-the-art methods in computer vision and successfully solved the challenges. As a result, the student introduced a full-pledged mobile application and made a paper published in a scientific conference of OISP students. Những thiếu sót chính của LVTN: - The writing is not very good, making a number of grammatical errors.
Đề nghị: Được bảo vệ □ Bổ sung thêm để bảo vệ □ Không được bảo vệ □ 9. 3 câu hỏi SV phải trả lời trước Hội đồng: a. Đánh giá chung (bằng chữ: giỏi, khá, TB): Điểm : 8. Quản Thành Thơ TRƯỜNG ĐẠI HỌC BÁCH KHOA CỘNG HÒA XÃ HỘI CHỦ NGHĨA VIỆT NAM KHOA KH & KT MÁY TÍNH Độc lập - Tự do - Hạnh phúc ---------------------------- Ngày 06 tháng 8 năm 2021 PHIẾU CHẤM BẢO VỆ LVTN (Dành cho người hướng dẫn/phản biện) 1.
Họ và tên SV: Lê Bá Thành MSSV: 1552340 Ngành (chuyên ngành): Khoa học Máy tính 2. Đề tài: Development of a mobile application to process house ownership certificate 3. Họ tên người hướng dẫn/phản biện: Bùi Hoài Thắng 4. Tổng quát về bản thuyết minh: Số trang: Số chương: Số bảng số liệu Số hình vẽ: Số tài liệu tham khảo: Phần mềm tính toán: Hiện vật (sản phẩm) 5.
Tổng quát về các bản vẽ: - Số bản vẽ: Bản A1: Bản A2: Khổ khác: - Số bản vẽ vẽ tay Số bản vẽ trên máy tính: 6. Những ưu điểm chính của LVTN: - Presented the motivation and then the problem of recognizing the text in the House Ownership Certificate to help the buyer locate the actual location of the selling house/land in order to avoid fraud. - Showed a study of background knowledge for that area, including some deep learning and artificial intelligence techniques, particularly in image processing,text detection and character recognition. - Proposed a model of combining those techniques in making a mobile system to help the buyers take the photo of the House Ownership Certificate and show the actual house on the Google Map, with Street View option.
Những thiếu sót chính của LVTN: 8. Đề nghị: Được bảo vệ X Bổ sung thêm để bảo vệ Không được bảo vệ 9. 3 câu hỏi SV phải trả lời trước Hội đồng: a. Đánh giá chung (bằng chữ: giỏi, khá, TB): Giỏi Điểm : 9.0 /10 Ký tên (ghi rõ họ tên) Bùi Hoài Thắng DECLARATION OF AUTHORSHIP I declare that this thesis and all the content, except the references from related projects, are the work of my own and have never been used at any other schools.
HO CHI MINH CITY, JULY, 2021. ACKNOWLEDGEMENTS I would like to express our deep gratitude and appreciation to Assoc. Quản Thành Thơ, My instructor for his guidance and instructions throughout this project. He never stopped chal- lenging me, and helping me develop my ideas with patience and advice.
My appreciation also extends to the teachers in the Faculty of Computer Science and Engineering in particular and all the teachers in the Ho Chi Minh city University of Technology in general. LE BA THANH vi PREFACE In this era, own houses or lands are becoming popular in many individuals’ plans, but not many have sufficient time to follow the seller to the house or land location to see the size in reality. Instead of that, with this application, just a few touches and choose users can obtain the house’s location by the house ownership diploma compared to an hour conversation to get a house’s location. According to Pham Thanh Hung or Shark Hung in the show Shark Tank VietNam, he is chairman of the joint-stock company CENINVEST who said about the problem of site viewing that employee has received a house ownership certificate in place and when onsite it was an- other place.
Because of the misleading in the coordinate of the house in reality. This thesis proposed a mobile application that can process the house ownership location based on the credential’s coordinate table. Furthermore, combine with the information on the type of house/land, method, type of use, and area to infer the sufficient in borrowing the money from bank to invest. The opportunity of that land could be upgraded from the farmland to resi- dential land by spending money to change the type of that house in the government base.
LIST OF FIGURES 1.1 House owner certificate sample .1 A normal pipline in OCR process .2 Use case of Google Vision API .3 The result of text processed and the return JSON encoded .4 Schematic of CRAFT network architecture .5 A normal pipline in OCR process .7 the filter 3*3 matrix with the image matrix 5*5 create a feature map 3*3 .8 The matrix after padding by 1 .9 The matrix with padding 1 with stride 1 and padding 1 with stride 2 .10 Max pooling with stride 2 and kernel 2 * 2 .11 CNN architecture in detect handwritting number .13 CNN flatten out the feature maps .1 React Native Architecture .2 Machine Learning libraries in Python .3 Client-Server Model .4 Old server model .5 Virtualization server model .6 Containerlization server model .7 The morphological dilation on binary image .8 The morphological erosion on binary image .9 Matrix inside diamond’s kernel structure element .1 Overall architecture OCR process .1 House owner certificate left side .2 House owner certificate right side .3 Orginal left side certificate .4 Binary image after threshold .5 All the contour detected on the image .6 Image text detection orginal and after using GDBSCAN .7 The raw text of Tesseract has processing .8 Image text detection hinh1 full page and after using GDBSCAN .9 House owner certificate right side .10 Image with text enhance and background noise removed .11 verticle binary image and horizontal binary image .12 The combined horizontal and vertical edge image .13 The joint node between horizontal and vertical line with dilation. 55 LIST OF FIGURES viii 5.14 All the contour extracted by the blending image and the table coordinate ex- tracted by Joint value .15 Main Screen and Upload Screen .16 Upload Screen before and after process .17 Review screen and map screen .1 response from backend API with all the field extracted as JSON .2 Curve text image processed by normal pipeline with GDBSCAN .3 Curve text image processed by Refine net model CRAFT .4 The polygon bounding text with the crop polygon .5 Result after rectify the polygon text. 72 LIST OF TABLES 1.1 Table types of land appear inside the house ownership certificate .1 Table accuracy tested by the author of VietOCR .1 Table testing Hinh1 with two module VietOCR and Tesseract .3 Table testing with different size of Hinh1 .2 Table summarize the information need to take out with Regular expression rule 62 6.3 Table evaluation the testing recognition data over the dataset with CPU and GPU 63 6.4 Table evaluation the proposed method over the dataset. 64 LIST OF ABBREVIATIONS The following list is the list of abbreviations which will be used later in this propose thesis.
UI User Interfaces WSGI Web Server Gateway Interface AI Artificial Intelligence CNN Convolution Neural Network CRAFT Character Region Awareness For Text Detection OCR Optical Character Recognition RN React Native CONTENTS Preface vi List of Figures vii List of Tables ix Abbreviations x Chapter1 OVERVIEW 1 1.2 Motivation for this work .3 Objective and Research scope. 6 Chapter2 RELATED WORK 8 2.1 Google Cloud Vision OCR .2 CRAFT Character Region Awareness for text detection .3 Convolution neural network .4 Vietnamese Optical Character Reader .1 Attention Optical Character Reader .2 Transformer Optical Character Reader. 19 Chapter3 THEORETICAL FRAMEWORK 20 3.2 Back-end Programming Language .3 Client-Server Model. 32 Chapter4 SYSTEM DESIGN 34 4.4 Left side certificate process part .5 Right side certificate process part.
56 Chapter6 EXPERIMENT AND EVALUATION 60 6.1 Block text extract with morphology .2 Table extract with morphology. 64 Chapter7 SUMMARY WORK 68 7. 71 Content A WORKING PLAN 72 Bibliography 73 List of keywords 74 1 OVERVIEW In this chapter, I will illustrate the contents of this topic and the structure of thesis proposed.2 Motivation for this work .3 Objective and Research scope .1 Introduction The real estate these days in Viet Nam is development and self-proved that is a conspicuous market more than ever, and this market makes a passive profit for an investor since every second this property can gain more money by itself. Not only that, this potential business seems to engage a massive amount of people who want to invest in and the value of the transaction related to this particular field earns slightly increase every day.
Especially true in the city’s high pace of life like Thanh Pho Ho Chi Minh, Ha Noi, and Da Nang, known as the leading-edge economy and the special economic zone Phu Quoc, Van Don, and Bac Van Phong where have the price and heat evaluation continuously rising every day.