The University of Texas at Austin
Computer Science Department

Computer Science 395T
Numerical Optimization for Graphics and AI

Fall 2017

General Information:

Time: Tuesdays and Thursdays 3:30PM-5:00PM
Place: GDC 4302
Instructor: Qixing Huang
Office hour: Fridays 3pm-5pm at GDC 5422.

This course will cover a wide range of topics in numerical optimization. The major goal is to learn a set of tools that will be useful for research in Artificial Intelligence and Computer Graphics. The course is a graduate-level course that combines instruction of basic material, written homeworks , and a final project. The course material integrates the theory of optimization and concrete real applications. Grading is based on homeworks (50%) and the final project (50%).

A partial list of applications to be covered:

  • Iterative closest point method for rigid and non-rigid registration.
  • Robust image and shape denoising/smoothing.
  • Scalable geometry reconstruction.
  • Convex relaxations for MAP inference.
  • Stochastic gradient descent for optimizing neural networks.
  • Convex and non-convex optimizations for low-rank matrix recovery.
  • Policy gradient methods.

    • Prereqs: The course assumes a good knowledge of linear algebra and probability. Please talk to me or email me if you are unsure if the course is a good match for your background.

    Textbook: Numerical Optimization.

    Schedule:

    Date Topics Reading Notes
    August 31th Introduction
    September 5th Linear Algebra Introduction to linear algebra. Homework 1.
    September 7th Probability Basic concentration bounds.
    September 12th Fundamentals of Unconstrained Optimization
    September 14th Line Search
    September 19th Line Search Applications
    September 21th Trust Region Methods I (Sub-problem) Homework 1 due. Homework 2 out.
    September 26th Trust Region Methods II (Global Convergence)
    September 28th Trust Region Methods (Applications)
    October 3th Conjugate Gradient Methods (Linear)
    October 5th Conjugate Gradient Methods (Nonlinear) Homework 2 due. Homework 3 out.
    October 10th Proximal Gradient Methods
    October 12th Theory of Constrained Optimization
    October 17th Optimality Condition Final project proposal Due.
    October 19th Linear Programming (Simplex Method) Homework 3 due. Homework 4 out.
    October 24th Linear Programming (Simplex Method II and Interior Point Method I)
    October 26th Linear Programming (Interior Point Method II)
    October 31th Quadratic Programming (Algorithms and Applications)
    November 2nd Guest Lecture
    November 7th Quadratic Programming II (Algorithms and Applications)
    November 9th Penalty, Augmented Lagrangian and SDP
    November 14th Spectral Methods I Homework 4 due. Homework 5 out.
    November 16th Spectral Methods II
    Novmeber 21th Topics in Convex Optimization I (Compressive Sensing)
    November 28th Topics in Convex Optimization II (Low-rank Matrix Recovery)
    November 30th Topics in Non-Convex Optimization I (Low-rank Matrix Recovery)
    December 5th Topics in Non-Convex Optimization II (Deep Neural Networks) Homework 5 due.
    December 7th Topics in Non-Convex Optimization III (Reweighted Least Squares) Final project report due.

    Final Project:

    The final project is done in groups of 2-3 students. Each project should have an initial proposal, a final report, and a final poster presentation. The project proposal shall describe four key components of a research project (namely Motivation, Technical Merit, Broader Impact, and Project Plan). The final report should be written as an academic research article. A more detailed instruction will be given later.