VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY HO CHI MINH UNIVERSITY OF TECHNILOGY FALCULTY OF COMPUTER SCIENCE AND ENGINEERING ——————– * ——————– REPORT CAPSTONE PROJECT 3D POINT CLOUD RECONSTRUCTION Major: COMPUTER SCIENCE THESIS COMMITTEE: COMPUTER SCIENCE - 02 CLC SUPERVISOR: NGUYEN DUC DUNG REVIEWER: LE THANH SACH ——o0o—— STUDENT: NGUYEN PHUOC NGUYEN PHUC - 2053342 HO CHI MINH CITY, JUNE 2024 Declaration As the author of this work, Nguyen Phuoc Nguyen Phuc - 2053342, hereby declare that the thesis entitled ”3D Point Cloud Reconstruction” represents my original work and findings, conducted under the supervision of Dr. Nguyen Duc Dung. We acknowledge that: • The work presented in this thesis, including the introduction, current work survey, baseline development, and experimental analysis, is based on my independent research conducted at Ho Chi Minh University of Technology. • All references, sources of information, and contributions from other researchers or sources have been appropriately cited and acknowledged in accordance with academic conventions and citation guidelines.
• Any assistance, technical support, or guidance received during the course of this research project has been duly acknowledged in the thesis. • The results, discussions, and conclusions drawn in this thesis are the outcome of my analysis and interpretation of the data gathered during this study. • The thesis has not been previously submitted for any degree or qualification at this or any other institution. Ho Chi Minh, JUNE 2024 Author Nguyen Phuoc Nguyen Phuc Thanks We would like to express my heartfelt gratitude for your unwavering support through- out the journey of my project.
Your guidance and mentorship have been invaluable, and your constant inspiration, feedback, and direction have truly made a significant differ- ence. We are deeply thankful for your dedication to my success, and we couldn’t have accomplished this without your help. we also want to extend my appreciation to all the other professors and teachers who have equipped us with the knowledge and skills necessary to carry out this project. Your dedication to imparting knowledge and fostering a love for learning has been instrumen- tal in my academic growth.
Additionally, we want to thank our friends who have provided valuable insights and ideas, contributing to the development of this project. Your willingness to share your thoughts and collaborate with us has been a source of encouragement and inspiration. This project would not have been possible without the collective efforts, encourage- ment, and guidance of all these individuals. we are sincerely grateful for their support, and we look forward to continuing to learn and grow under their mentorship and guid- ance.
Once again, thank you from the bottom of my heart. Content Sumary Introduction (Chapter 1):This chapter motivates the research on point cloud comple- tion, outlining the importance of accurate 3D object reconstruction from incomplete data. It defines point cloud missingness and discusses its causes, highlighting the real-world applications where this problem arises. Background Knowledge (Chapter 2):This chapter provided the essential background for understanding 3D point cloud completion.
This foundation prepares you for the deeper dive into the mechanisms and advancements of point cloud completion in the following sections. Survey (Chapter 3 + Chapter 4):These 2 chapters provide a comprehensive survey of existing point cloud completion methods. It covers different datasets, evaluation metrics (Chapter 3), and previously proposed approaches (Chapter 4). A comparative analysis of various methods highlights their strengths and weaknesses, offering insights into the current state-of-the-art.
Baseline Method (Chapter 5):This chapter focuses on the chosen baseline method, AdaPoinTr, for further investigation. It provides a detailed description of the architec- ture, including the encoder-decoder structure, multi-head attention mechanism, positional encoding, and other key components. Improve The Baseline Method (Chapter 6 + Chapter 7):These 2 chapters shed light on the limitations of existing baseline models for 3D point cloud completion. We explored the challenges these models face when dealing with incomplete and noisy data, which are hallmarks of real-world scenarios.
To bridge this gap and enable application to real-life problems, we then proposed advancements that focus on enhancing the model’s ability to handle imperfect data and generalize effectively to unseen real-world situations. These advancements aim to pave the way for robust and reliable point cloud completion in practical applications. Experiments and Results (Chapter 8):This chapter presents the experimental results of applying AdaPoinTr to various datasets. It analyzes the performance of the model in terms of accuracy and robustness, comparing it to other methods and highlighting its strengths and limitations.
The results offer valuable insights into the effectiveness of AdaPoinTr and pave the way for further research and development. Conclusion (Chapter 9):This chapter summarizes the key findings of the thesis, em- phasizing the contributions of the point cloud completion research. It discusses the limi- tations of the current approach and outlines potential avenues for future exploration. This chapter concludes by reiterating the importance and significance of point cloud comple- tion in various computer vision applications.1 The Rise of 3D Data and Point Clouds .2 Challenges of Raw Point Cloud Data .3 3D Point Cloud Reconstruction: A Solution .4 Project Scope: Point Cloud Completion .5 Project Goals and Deliverables .1 Fundamentals of 3D Point Clouds .1 What is 3D Point Cloud .3 Point Cloud Processing Techniques .2 Machine Learning and Deep Learning in 3D Point Cloud.
10 3 Datasets and Metrics for 3D Point Cloud 13 3.1 ShapeNet [14] - A Large-Scale Dataset for 3D Shape Understanding 13 3.2 ModelNet [18]: A Clean and Categorized Dataset for 3D Point Cloud Analysis .3 PCN [21] Dataset: A Benchmark for Point Cloud Completion .4 S3DIS [19]: Unveiling the Structure of Indoor Scenes .2 Earth’s Mover Distances .1 Point-based methods .2 Convolution-based methods .3 Graph-based methods .4 GAN-based method .5 Transformer-based methods. 26 i 5 AdaPoinTr: Diverse Point Cloud Completion with Adaptive Geometry-Aware Transformers 28 5.1 Why chosing AdaPointr for 3D Point Cloud Completion .1 Set-to-Set Translation with Transformers .3 Geometry-aware Transformer Block .5 Multi-Scale Point Cloud Generation .6 Adaptive Denoising Queries. 34 6 Enhancing the Existing Loss 36 6.1 Drawbacks of Chamfer Distance Loss .2 Solution: InfoCD [55] - A Contrastive Chamfer Distance Loss .1 Core Concept Of InfoCD .2 Chamfer Distance Loss .3 InfoCD Loss Function. 39 7 Multi-Object Completion 42 7.1 The Limitation of Synthetic Training .2 AdaPoinTr Initial Setting .3 Limited Generalize ability .2 A more general learning .1 The Partial Normalize .2 Anti One Direction .3 Uniform Random Sampling .3 Multi-ShapeNet Dataset .1 Steps to create ShapeNet Room .4 Transfer Learning on S3DIS datset .1 Object Point Cloud Completion .1 Benchmark for Diverse Point Completion .2 Results on PCN dataset .3 Results on ShapeNet34 .4 Result on ShapeNet55 dataset .2 Multi Object Completion .1 Setting for Multi Object Completion .2 Experiments result on Multi ShapeNet .3 Experiment on S3DIS.
67 9 Conclusion And Future Research Direction 77 9.2 Future Research Direction. 78 List of Tables 3.1 Summary of existing datasets for point cloud completion [5].1 Results on the PCN dataset [15].2 Results on the ShapeNet55 dataset [15].3 Results on the ShapeNet34 dataset [15].4 Complexity analysis of existing methods.1 Result on PCN dataset after 100 epochs.2 Result on PCN dataset (InfoCD) after 100 epochs.3 Result on ShapeNet34 dataset (CD) after 10 epochs.4 Result on ShapeNet Unseen 21 (CD) after 10 epochs.5 Result on ShapeNet34 dataset (InfoCD) after 10 epochs.6 Result on ShapeNet21 Unseen (InfoCD) after 10 epochs.7 Result on ShapeNet55 dataset (CD) after 10 epochs (part 1).8 Result on ShapeNet55 dataset (CD) after 10 epochs (part 2).9 Result on ShapeNet55 dataset (InfoCD) after 10 epochs (part 1).10 Result on ShapeNet55 dataset (InfoCD) after 10 epochs (part 2).11 Test result on Multi ShapeNet 200 epoch with Chamfer Distance.12 Test result on Multi ShapeNet 200 epoch with Info Chamfer Distance. 70 iv List of Figures 1.1 3D Point Cloud for Autonomous Driving Car.2 3D Point Cloud for Robotic .3 Example building point cloud after floor partitioning [4].4 Schematic of complete point cloud and missing 70% point cloud [5].5 Reasons for incomplete point clouds [5].1 Convolutional Neural Network for Images [10].2 Encoder-Decoder Transformer Architecture [14].3 An illustration of a graph-based network [17].1 Some categories in ShapeNet dataset [15].1 End-to-end network for point clouds completion. N represents the dimen- sion of latent space [5].2 Description of two-step-folding decoding.3 The architecture of FoldingNet [35].2 Comparisons of the vanilla Transformer block and the proposed geometry- aware Transformer block [14].3 Three types of queries for transformer decoder [15].4 Improvement made by AdaPoinTr compare to PoinTr [15].1 Illustration of comparison among CD, MI, and InfoCD with different numbers of samples.2 Illustration of the moving directions of matched points.1 Some examples from ShapeNet.
It can be seen that all object, despite their size, is all at a same scale and direction.2 Examples when applying original model objects that are out of distibution.3 Some Rooms example from Multi-ShapeNet.4 Some rooms that collected from S3DIS dataset.5 Some objects that collected from S3DIS dataset.1 Inference on PCN after 100 epochs.2 Inference result on ShapeNet34 (easy missing).3 Inference result on ShapeNet21 (easy missing).4 Inference result on ShapeNet34 and 21 with 25% missing points.5 Point Spreading Comparing between a) CD and b) InfoCD.6 Inference result on ShapeNet55 (easy missing).7 Multi-ShapeNet room View 1. a) Full room before cutting. b) After re- moveing 50% point and c) After completion process using AdaPoinTr.8 Multi-ShapeNet room View 2. a) Full room before cutting.
b) After re- moveing 50% point and c) After completion process using AdaPoinTr.9 Multi-ShapeNet inference results.10 Transfer inferencing on S3DIS. The left image is raw object, the right one is after completion.1 The Rise of 3D Data and Point Clouds In recent years, the world of technology has witnessed an incredible surge in the de- velopment of 3D technologies, leading to remarkable advancements in various fields. The applications of 3D technology have transcended the realm of imagination, permeating our daily lives, and revolutionizing industries such as autonomous vehicles, architecture, gam- ing, and more. These innovations have transformed the way we interact with the world and opened new avenues for creativity and problem-solving.
(a) Illustration of 3D Point cloud segmentation following the road slope. Ground points are green, obstacles are pink [1]. (b) Examples of autonomous vehicles. In all model, the Lidar sensor can be seen on the roof of the car [2].1: 3D Point Cloud for Autonomous Driving Car.2: 3D point cloud (colored) with pose estimates (grey) of OP-Net AP on real- world data without ICP refinement for ring screws.
The emergence of 3D technology has been particularly influential in the development of autonomous vehicles, where it enables precise environmental mapping and navigation. Architects and urban planners now utilize 3D models to visualize and design structures with unparalleled precision. Gamers are immersed in lifelike virtual worlds, thanks to the realistic 3D environments created for their enjoyment. These are just a few examples of how 3D technology has redefined our experiences and possibilities.
Within the ever-evolving landscape of 3D technology, point clouds stand as a revo- lutionary force, silently gathering and transforming the intricate details of our physical world. These dense collections of data points act like digital detectives, meticulously cap- turing the nuances of objects, spaces, and environments. They are more than just dots on a screen; they are the raw material for a new era of understanding and interacting with the world around us. One of the most potent applications of point clouds lies in 3D scanning.
Imagine holding a wand that, instead of casting spells, captures the essence of reality in a digi- tal tapestry of points. This is the power of 3D scanners equipped with laser or LiDAR technology. They sweep their beams across objects and landscapes, transforming every bump, curve, and crevice into a precise numerical map. From the weathered facade of an ancient temple to the intricate machinery of a modern factory, point clouds breathe digital life into the physical world.3: Example building point cloud after floor partitioning [4].
But point clouds are not mere static representations. They are the building blocks for a dynamic digital realm.
