Qixing Huang

Maps between sets are fundamental mathematical quantities. Depending on how we define sets, maps can take various forms. Examples include correspondences between image pixels, correspondences between graph vertices, and rigid transformations between 3D protein structures. Map computation is challenging because there may be insufficient or sometimes biased signals between a pair of objects/domains for recovering the underlying map. Map synchronization, which jointly optimizes maps among a collection of relevant objects/domains, provides a highly effective way to address this issue. In this context, there exists a natural regularization called \textsl{cycle-consistency} or \textsl{path-invariance}, which states that composite maps along cycles should be the identity map or composite maps between two objects/domains are invariant to the paths that connect them. Intuitively, this constraint allows us to convert the difficult task of computing maps between two dissimilar objects into an easy task by composing maps along a path of similar object pairs. A fundamental challenge of map synchronization is to leverage the cycle-consistency constraint for map computation effectively.

I got interested in this problem in [SIGGRAPH06], which needs to solve a map synchronization problem to reconstruct a complete object from its broken pieces. Our [SGP13] paper established an equivalence between the cycle-consistency constraint and simple properties of the data matrix that encodes pairwise maps in blocks. This leads to a semidefinite programming formulation for optimizing consistent correspondences across object collection. The approach possesses the first tight determinisc recovery guarantee. In series of papers, we extend this approach in multiple ways, including partial maps [ICML14], consistent functional maps [ICCV13][CVPR14][SIGGRAPH14], spectral formulations [NeurIPS16][ICML18], non-convex formulation [NeurIPS17][CVPR19b], and tensor formulation [SIGGRAPH19].

A very interesting scenario is that objects possess self symmetries, meaning the map between two objects is not unique. In [ECCV18], we developped a data representation that enables joint optimization of symmetry groups and maps between symmetric objects. In , we generalized the diffusion-and-clustering scheme in and developed an iterative approach for synchronizing transformations across multiple objects with provable guarantees.

Consistent maps across image and shape collections enable fruitful applications, including joint object segmentation [ICCV13][CVPR14][SIGGRAPH14][SIGGRAPH19], geometry reconstruction [SIGGRAPH06][CVPR19b], multi-view 3D human pose estimation [CVPR19c], clustering [ICML18][SIGGRAPH19], and symmetry detection [ECCV18].

Map synchronization also enjoys rich applications in the deep learning era. Neural networks are maps between source and target domains. In the context of a graph of neural networks across multiple domains (e.g., hybrid 3D representations), the combinatorial graph constraints provide unsupervised training losses for joint learning of neural networks. Since one cannot encode neural networks as matrices, the constrained matrix recovery formulations are not applicable anymore, and one has to move back to the original combinatorial constraints. In [CVPR16], we demonstrated the effectiveness of self-supervision along one cycle. In [NeurIPS19], we introduced cycle-consistency bases for joint learning of an undirected graph of neural networks. In [CVPR19], we defined path-invariance bases and introduced an algorithm for computing a polynomial-size path-invariance basis for a directed graph of neural networks.

Our ICCV 2021 paper applies the concept of map synchronization to fuse absolute and relative attributes for the task of scene synthesis.

[NeurIPS23] Jiaxin Lu, Yifan Sun and Qixing Huang. Jigsaw: Learning to Assemble Multiple Fractured Objects. Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS) 2023.

[ARXIV23] Haitao Yang, Xiangru Huang, Bo Sun, Chandrajit Bajaj and Qixing Huang. 3D Feature Prediction for Masked-AutoEncoder-Based Point Cloud Pretraining. arXiv preprint arXiv:2304.10523

[CVPR23] Yifan Sun and Qixing Huang. Pose Synchronization Under Multiple Pair-Wise Relative Poses. Computer Vision and Pattern Recognition (CVPR) 2023

[ICCV21] Haitao Yang, Zaiwei Zhang, Siming Yan, Chongyang Ma, Haibin Huang, Yi Zheng, Chandrajit Bajaj, and Qixing Huang. Scene Synthesis via Uncertainty-Driven Attribute Synchronization. International Conference on Computer Vision (or ICCV) 2021.

[NeurIPS19] Leonidas Guibas, Qixing Huang. and Zhenxiao Liang. A Condition Number for Joint Optimization of Cycle-Consistent Networks. Advances in Neural Information Processing Systems(NIPS), 2019 (Spotlight Presentation). (Authors by alphabetical order)

[ICCV19] Yifan Sun, Jiacheng Zhuo, Arnav Mohan, and Qixing Huang. K-Best Transformation Synchronization. International Conference on Computer Vision' 2019.

[SIGGRAPH19] Qixing Huang, Zhenxiao Liang, Haoyun Wang, Simiao Zuo, and Chandrajit Bajaj. Tensor Maps for Synchronizing Heterogeneous Shape Collections. ACM Transaction on Graphics 38(4) (Proc. Siggraph 2019).

[CVPR19a] Zaiwei Zhang, Zhenxiao Liang, Lemeng Wu, Xiaowei Zhou, and Qixing Huang. Path-Invariant Map Networks. Computer Vision and Pattern Recognition (or CVPR) 2019. Oral Presentation and Best Paper Finalist .

[CVPR19b] Xiangru Huang, Zhenxiao Liang, Xiaowei Zhou, Yao Xie, Leonidas Guibas, and Qixing Huang. Learning Transformation Synchronization. Computer Vision and Pattern Recognition (or CVPR) 2019.

[CVPR19c] Junting Dong, Wen Jiang, Qixing Huang, Hujun Bao, and Xiaowei Zhou. Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views. Computer Vision and Pattern Recognition (or CVPR) 2019.

[ECCV18] Yifan Sun*, Zhenxiao Liang*, Xiangru Huang* and Qixing Huang. Joint Map and Symmetry Synchronization. European Conference on Computer Vision 2018 (or ECCV 2018).

[ICML18] Chandrajit Bajaj, Tingran Gao, Zihang He, Qixing Huang and Zhenxiao Liang. Simultaneous Mapping and Clustering via Spectral Decompositions. International Conference on Machine Learning' 2018. (Authors by alphabetical order)

[CVPR18] Nan Hu, Qixing Huang,Boris Thibert and Leonidas Guibas. Distributable Consistent Multi-Object Matching. Computer Vision and Pattern Recognition' 2018 (Spotlight Presentation).

[NeurIPS17] Xiangru Huang^*, Zhenxiao Liang^*, Chandrajit Bajaj and Qixing Huang. Translation Synchronization via Truncated Least Squares. Neural Information Processing Systems (or NeurIPS) 2017. Spotlight Presentation. (*indicates equal contribution). Code

[NeurIPS17] Yanyao Shen, Qixing Huang, Nathan Srebro, and Sujay Sanghavi. Normalized Spectral Map Synchronization. Neural Information Processing Systems (or NeurIPS) 2016.

[CVPR 16] Tinghui Zhou, Philipp Kr\E4henb\FChl, Mathieu Aubry, Qixing Huang, and Alexei A. Efros. Learning Dense Correspondence via 3D-guided Cycle Consistency. IEEE Conference on Computer Vision and Pattern Recognition 2016 (Oral Presentation).

[SIGGRAPH14] Qixing Huang, Fan Wang, and Leonidas Guibas. Functional Map Networks for Analyzing and Browsing Large Shape Collections . ACM Transaction on Graphics 33(4) (Proc. Siggraph 2014). Code

[CVPR14] Fan Wang, Qixing Huang, Maks Ovsjanikov, and Leonidas Guibas. Unsupervised Multi-Class Joint Image Segmentation. IEEE Conference on Computer Vision and Pattern Recognition 2014.

[ICML14] Yuxin Chen, Leonidas Guibas, and Qixing Huang. Near-Optimal Joint Object Matching via Convex Relaxation. International Conference on Machine Learning 2014.

[SGP13] Qixing Huang and Leonidas Guibas. Consistent Shape Maps via Semidefinite Programming. Computer Graphics Forum, Volume 32(5) (Proc. Symposium on Geometry Processing ). (Best Paper Award). Code

[ICCV13] Fan Wang, Qixing Huang, and Leonidas Guibas. Image Co-Segmentation via Consistent Functional Maps. . International Conference on Computer Vision 2013.

[SIGASIA12] Qixing Huang, Guoxin Zhang, Lin Gao, Shimin Hu, Adrian Butscher, and Leonidas Guibas. An Optimization Approach for Extracting and Encoding Consistent Maps in a Shape Collection. ACM Transactions on Graphics 31(6) (Proc. SIGGRAPH Asia 2012)

[SIGGRAPH06] Qixing Huang. Simon Fl#246ry, Nathsha Gelfand, Michael Hofer, and Helmut Pottmann. Reassembling Fractured Objects by Geometric Matching. ACM Transactions on Graphics 25(3) (Proc. SIGGRAPH 2006).