Graph learning for point cloud generation and reconstruction

Abstract

As the most general and fundamental representation of 3D objects, a point cloud provides massive fexibility in geometric structure and scale. Point clouds are applicable to a wide variety of science and engineering fields from medical imaging to design and visualization. However, learning from point cloud data is challenging when applying traditional learning methods. Specifcally, unsupervised point cloud generation and reconstruction in three-dimensional space are sensitive to irregularity and sparsity of data points, even after the remarkable achievements in classifcation and semantic segmentation methods presented in the current literature. Fortunately, topological graph representations, as structural information repository for point clouds, provide the ability to exploit the latent representations of 3D objects. A graph maintains the topological knowledge of structured data, allowing relational information about different entities to be preserved. In other words, graph structures of the features of a point cloud make it easier to be analyzed by learning models. This gives us the opportunity to regard a graph as the topological representation for a descriptive point set and unfold it using deep learning frameworks. In this thesis, we focus on 3D point cloud estimation and present a class of graph-based learning models for applications ranging from point cloud generation to dense reconstruction. Different from state-of-the-art point cloud learning models, our approaches rely on gradually formulating topological graphs that embed the structure of the potential 3D shape. The specifc contributions presented in this thesis are as follows. First, we design an adversarial learning framework to incorporate hierarchical graph inference into the deep learning model. To deploy the topology information of the entire graph in complex shape generation, we accommodate self-attention masking and spectral Graph Convolution Network (GCN) to tree architecture. The spectral analysis based on Graph Signal Processing (GSP) makes our model more interpretable in exploiting graph topology. Second, we improve this Generative Adversarial Network (GAN) and further exploit its potentiality in topological structure generation. Specifcally, we examine the correlation between latent graph topology and their corresponding 3D partial structures. Third, we propose a novel auto-encoding architecture aiming at learning 3D shapes accurately from sparse point clouds. On the decoder side, a new attention-based mechanism is presented to better take advantage of the topology of the latent representation. With this method, our model extends the receptive field of each node and associates distinguishable local features with global topology information. The common theme of these models is analyzing graphical features and evolving them to various 3D geometry represented by point clouds.