Qixing Huang

Enforcing Geometric, Physical, and Topological Priors for 3D Generation 

This is a central direction in my group. Thanks to advances in generative models, such as GAN, VAE, AD, NF, and diffusion models, for images/videos, there is great interest in extending these models to generate 3D content. However, most existing 3D generation approaches have focused on adopting a 2D generator under a suitable 3D representation, e.g., 3D grids, meshes, point clouds, and implicit surfaces. The typical scheme is distribution alignment. With limited training data, distribution alignment typically incurs implicit regularizations. Although the effects of implicit regularizations depend on the chosen 3D representation, none of the existing 3D representations can preserve 3D geometric, physical, and topological priors. As neural networks are over-parameterized, when learning 3D generators, it is essential to instill priors as regularization terms to enhance the generalizability of the resulting generators.

Since 2021, my group has been looking into this direction for deformable shape generators. Existing deformable shape generations, based on distribution alignment, cannot ensure that the underlying deformation between adjacent shapes preserves geometric structures, e.g., as-rigid-as-possible (ARAP) where the deformation is locally rigid. In [ICCV21], we developed an ARAP regularization loss that can enforce such priors. This work also indicates a Riemannian metric for deformable shape generators. In [SIGA23], we used this metric to design the latent space of deformable shape generators so that linear interpolations in latent spaces follow approximately geodesic curves and different axes disentangle pose and shape variations.

Another exciting idea is that when enforcing geometric regularizations on deformable shape generators, we can perform map synchronization that unifies pair-wise matching and joint-shape matching, leading to high-quality inter-shape correspondences. This work [Arxiv23] has been accepted to ICLR 2024.

Besides deformation priors, my group has multiple projects on enhancing the physical stability and topological generalization of man-made shape generators. A recent paper CVPR24 that studies this topic was accepted as an oral paper at CVPR 2024. Another recent paper ICML24 that studies how to enhance topological generalization of man-made generators was accepted at ICML 2024.