The University of Texas at Austin
Computer Science Department

GAMES Advanced Course
3D Reconstruction and Understanding

Summer 2021

General Information:

Time: Fridays 10:00am-11:50am on GAMES (Beijing Time).
Place: http://webinar.games-cn.org/
Instructor: Qixing Huang

This course will cover a broad range of topics in the general area of 3D Vision and 3D Geometry Processing, ranging from 1) reconstructing 3D models from images and depth scans, 2) 3D representations (e.g., for neural networks), and 3) analysis and processing of 3D models. An unique characteristics of this course is that we will install the basic theory of numerical optimization throughout. The course is a graduate-level course that is self-contained.

Prereqs: The course assumes a good knowledge of linear algebra and probability.

Textbooks (Not Required but Recommended):

CS395T_Numerical_Optimization.jpg

[NW]: Numerical Optimization

CS395_3D_Vision.jpg

[MKSS]: An Invitation to 3D Vision

CS395_Polygon_Mesh_Processing.jpg

[BKPAL]:Polygon Mesh Processing

CS395T_Point_Based_Graphics.jpg

[GP]: Point-Based Graphics

Schedule:

Date Topics Reading Notes
July 9th (Reconstruction): Introduction and Scanning
July 16th (Reconstruction): Registration trimesh2, multi-reg, Open3D
July 23th (Reconstruction): Surface Reconstruction Poisson Surface Reconstruction
July 30th (Reconstruction):Structure From Motion
August 6th (Reconstruction):Multi-View Stereo
August 13th (Reconstruction):Map Synchronization for Inverse Problems
August 20th (Representation): Point Cloud Processing
August 27th (Representation): Mesh Processing
September 10th (Deep Learning): 3D Deep Learning
September 17th (Deep Learning): Geometric Deep Learning I
October 7th (Deep Learning): Geometric Deep Learning II
October 14th (Deep Learning): Geometric Deep Learning III
October 22th (Deep Learning): Hybrid 3D Representations
October 29th (Deep Learning): Course Warp&Up

Course Reading (Being Updated):

Reading Group 1 (Scanning):

  • Real-Time 3D Model Acquisition Szymon Rusinkiewicz,Olaf Hall-Holt, and Marc Levoy. ACM SIGGRAPH 2002.
  • The 3D model Acquisition PipelineFausto Bernardini and Holly Rushmeier. Computer Graphics Forum 2002.
  • The digital Michelangelo Project: 3D Scanning of Large Statues. Marc Levoy, Kari Pulli, Brian Curless, Szymon Rusinkiewicz, David Koller, Lucas Pereira, Matt Ginzton, Sean Anderson, James Davis, Jeremy Ginsberg, Jonathan Shade, and Duane Fulk.

    • Reading Group 2 (Registration):

  • A Method for Registration of 3-D Shapes. Paul J. Besl and Neil D. McKay. IEEE Transactions on Pattern Recognition and Machine Intelligence. 1992.
  • Efficient Variants of the ICP Algorithm. Szymon Rusinkiewicz and Marc Levoy. 2001.
  • Geometry and Convergence Analysis of Algorithms for Registration of 3D Shapes. Helmut Pottmann, Qixing Huang, Yongliang Yang, and Shimin Hu. International Journal of Computer Vision. 2006.
  • "High Quality Pose Estimation by Aligning Multiple Scans to a Latent Map."Qixing Huang and Dragomir Anguelov. IEEE International Conference on Robotics and Automation 2010.
  • "DynamicFusion: Reconstruction and Tracking of Non-rigid Scenes in Real-Time."Richard Newcombe, Dieter Fox, and Steve Seitz. CVPR 2015.

    • Reading Group 3 (Surface Reconstruction):

  • Screened Poisson Surface Reconstruction.Michael Kazhdan AND Hugues Hoppe. ACM TOG 2013.
  • Surface Reconstruction from Unorganized Points. Hugues Hoppe, Tony DeRose, Tom Duchamp, John McDonald, and Werner Stuetzle. SIGGRAPH 1992.
  • The Power Crust Nina Amenta Sunghee Choi and Ravi Krishna Kolluri.
  • The Ball-Pivoting Algorithm for Surface Reconstruction. . F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, and G. Taubin. IEEE TVCG. 1999.
    		•

    Reading Group 4 (Structure-From-Motion):

  • Structure-From-Motion Revisited. Johannes L. Schonberger and Jan-Michael Frahm; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 4104-4113.
  • A Survey of Structure from MotionOnur Ozyesil, Vladislav Voroninski, Ronen Basri, and Amit Singer. Acta Numerica Acta Numerica. 2017.
    		•

    Reading Group 5 (Multi-View Stereo):

  • "Accurate, Dense, and Robust Multi-View Stereopsis.".Yasutaka Furukawa and Jean Ponce. PAMI, August 2010.
  • "DeepStereo: Learning to Predict New Views From the World's Imagery.".John Flynn, Ivan Neulander, James Philbin, and Noah Snavely. CVPR 2016. Presenter: Tianyu Cheng
    		•

    Reading Group 6 (Map Synchronization):

  • Angular Synchronization by Eigenvectors and Semidefinite Programming. Amit Singer. Applied and computational harmonic analysis 30 (1), 20-36. 2011.
  • Consistent Shape Maps via Semidefinite Programming. Qixing Huang and Leonidas Guibas. Symposium on Geometry Processing 2013.
  • Exact and Stable Recovery of Rotations for Robust Synchronization. Lanhui Wang and Amit Singer. Information and Inference: A Journal of the IMA, 2013
  • Functional map networks for analyzing and exploring large shape collections. Qixing Huang, Fan Wang, Leonidas Guibas. ACM Transactions on Graphics (TOG), 33(4), 2014.
  • Learning transformation synchronization. Xiangru Huang, Zhenxiao Liang, Xiaowei Zhou, Yao Xie, Leonidas J Guibas, Qixing Huang. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8082-8091, 2019.
  • Translation synchronization via truncated least squares. Xiangru Huang, Zhenxiao Liang, Chandrajit Bajaj, Qixing Huang. Advances in neural information processing systems, 1459-1468, 2017.

    • Reading Group 6 (Map Synchronization):

  • Angular Synchronization by Eigenvectors and Semidefinite Programming. Amit Singer. Applied and computational harmonic analysis 30 (1), 20-36. 2011.
  • Consistent Shape Maps via Semidefinite Programming. Qixing Huang and Leonidas Guibas. Symposium on Geometry Processing 2013.
  • Exact and Stable Recovery of Rotations for Robust Synchronization. Lanhui Wang and Amit Singer. Information and Inference: A Journal of the IMA, 2013
  • Functional map networks for analyzing and exploring large shape collections. Qixing Huang, Fan Wang, Leonidas Guibas. ACM Transactions on Graphics (TOG), 33(4), 2014.
  • Learning transformation synchronization. Xiangru Huang, Zhenxiao Liang, Xiaowei Zhou, Yao Xie, Leonidas J Guibas, Qixing Huang. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8082-8091, 2019.
  • Translation synchronization via truncated least squares. Xiangru Huang, Zhenxiao Liang, Chandrajit Bajaj, Qixing Huang. Advances in neural information processing systems, 1459-1468, 2017.

    • Reading Group 7 (Point Cloud):

  • Pointshop 3D: An Interactive System for Point-Based Surface Editing. Matthias Zwicker, Mark Pauly, Oliver Knoll, and Markus Gross. SIGGRAPH '02: Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Technique, July 2002 Pages 322–329.
  • Efficient Simplification of Point-Sampled Surfaces. Mark Pauly, Markus Gross, and Leif P. Kobbelt. IEEE Visualization 2001.
  • Shape Modeling with Point-Sampled Geometry. Mark Pauly and Richard Keiser and Leif P. Kobbelt, and Markus Gross. ACM Transactions on Graphics, Volume 22, Issue 3, July, pages 641–650, 2013

    • Reading Group 8 (Mesh Processing):

  • Polygonal Mesh Processing. Mario Bostch. Leif Kobbelt, Mark Pauly, Pierre Alliez, and Bruno Levy, 2010.