Learning to learn point signature for 3D shape geometry

Abstract

Point signature is a representation that describes the structural geometry of a point within a neighborhood in 3D shapes. Conventional approaches apply a weight-sharing network, e.g., Graph Neural Network (GNN), to all neighborhoods of all points to directly generate point signatures and gain the generalization ability of the network by extensive training over amounts of samples from scratch. However, such approaches lack the flexibility to rapidly adapt to unseen neighborhood structures and thus cannot generalize well to new point sets. In this paper, we propose a novel meta-learning 3D point signature model, 3D meta point signature (MEPS) network, which is capable of learning robust 3D point signatures. Regarding each point signature learning process as a task, our method obtains an optimized model over the best performance on the distribution of all tasks, generating reliable signatures for new tasks, i.e., signatures of unseen point neighborhoods. Specifically, our MEPS consists of two modules: a base signature learner and a meta signature learner. During training, a base-learner is trained to perform specific signature learning tasks. Meanwhile, a meta-learner is trained to update the base-learner with optimal parameters. During testing, the meta-learner learned with the distribution of all tasks can adaptively change the base-learner parameters to accommodate unseen local neighborhoods. We evaluate our MEPS model on 3D shape correspondence and segmentation. Experimental results demonstrate that our method not only gains significant improvements over the baseline model to achieve state-of-the-art performance, but also is capable of handling unseen 3D geometry. Our implementation is available at https://github.com/hhuang-code/MEPS.