CS395T: Numerical Optimization for Graphics and AI

The University of Texas at Austin
Computer Science Department

Computer Science 395T
Numerical Optimization for Graphics and AI (3D Geometry)

Spring 2021

General Information:

Time: Mondays and Wednesdays 3:30PM-5:00PM
Place: Zoom
Instructor: Qixing Huang
Office hour: Fridays 5pm-7pm on Zoom.

This course will cover a broad range of topics in the general area of 3D Vision and 3D Geometry Processing, ranging from 1) reconstructing 3D models from images and depth scans, 2) 3D representations (e.g., for neural networks), and 3) analysis and processing of 3D models. An unique characteristics of this course is that we will install the basic theory of numerical optimization throughout. The course is a graduate-level course that combines instruction of basic material, written homeworks , and a final project. The course targets for students who will conduct research in Graphics, Vision, Robotics, and Computational Biology .Grading is based on homeworks (70%) and the final project (30%).

Prereqs: The course assumes a good knowledge of linear algebra and probability. Please talk to me or email me if you are unsure if the course is a good match for your background.

Textbooks (Not Required but Recommended):

[NW]: Numerical Optimization

[MKSS]: An Invitation to 3D Vision

[BKPAL]:Polygon Mesh Processing

[GP]: Point-Based Graphics

Schedule (A: Application, T: Theory):

Date	Topics	Reading	Notes
Jan. 20th	(A): Introduction
Jan. 25th	(T): Math Review (Linear Algebra, Rotation, Quaternion)	Rotation Quaternion	Homework 1
Jan. 27th	(T): Fundamentals of Unconstrained Optimization	Chapter 2 of [NW]
Feb. 1th	(T): Fundamentals of Constrained Optimization	Chapter 12 of [NW]
Feb. 3th	(A): Image Formation	Chapter 3 of [MKSS]
Feb. 8th	(A): Image Primitives and Correspondence	Chapter 4 of [MKSS]
Feb. 10th	(A): Reconstruction from Two Calibrated Views	Chapter 5 of [MKSS]
Feb. 15th	(A): Camera Calibration and Self-Calibration	Chapter 6 of [MKSS]
Feb. 17th	(A): Introduction to Multiple View Reconstruction	Chapter 7 of [MKSS]	Homework 1 due. Homework 2 out.
Feb. 22th	(T): Line Search Techniques	Chapter 3 of [NW]
Feb. 24th	(T): Trust Region Methods	Chapter 4 of [NW]
Mar. 1st	(A): Image Matching and Bundle Adjustment	Chapter 14.3-14.4 of [MKSS]
Mar. 3th	(A): Multi-View Stereo
Mar. 8th	(A): RGB-D Based 3D Reconstruction	Chapter 14.5 of [MKSS]	Homework 2 due. Homework 3 out.
Mar. 10th	(T): Large-Scale Optimization (Proximal Gradient)	Chapter 13 of [NW]
Mar. 15th	(T): Large-Scale Optimization (ADMM)	Chapter 14 of [NW]
Mar. 17th	(T): 3D Representation I (PointCloud, Implicit)
Mar. 22th	(A): Midterm	Chapter 1-2 of [GP]	Homework 3 due. Homework 4 out.
Mar. 24th	(A): 3D Representation II (Parametric, Discrete Differential Geometry)	Chapter 1-2 and Chapter 7 of [BKPAL]
Mar. 29th	(A): 3D Representation III (Discrete Differential Geometry)
Mar. 31th	(A): 3D Representation IV (Triangular Mesh)
Apr. 5th	(A): 3D Representation V (Shape Deformation)
Apr. 7th	(A): Hybrid 3D Representation (cycle-consistency + path-invariance)
Apr. 12th	(A): 3D Deep Learning I (Understanding)		Homework 4 due. Homework 5 out.
Apr. 14th	(T): 3D Deep Learning II (Understanding)
Apr. 19th	(A): 3D Deep Learning IV (Synthesis)
Apr. 21th	(A): 3D Deep Learning IV (Synthesis)
Apr. 26th	(T): Over-parameteization in Deep Learning I
Apr. 28th	(T): Over-parameteization in Deep Learning II
May 3th	(T): Joint Learning across Geometric Data		Homework 5 due.
May 5th	Final Project Presentations		Final project report due.

Final Project:

The final project is done in groups of 2-3 students. Each project should have an initial proposal, a final report, and a final poster presentation. The project proposal shall describe four key components of a research project (namely Motivation, Technical Merit, Broader Impact, and Project Plan). The final report should be written as an academic research article. A more detailed instruction will be given later.