The University of Texas at Austin
|
Computer Science 378
|
|
Introduction:
This is an intro course in image and video generative models. It is intended for upper-level undergraduate students.
Visual generative models is an active field that deals with representations and algorithms that generate visual contents in the form of images, videos, animations, and 3D content. This is an active and fast evolving field across industrial and scientific domains. It builds on image representations, deep learning, and algrotihms for generative modeling. Our goal is to teach basics from scratch towards writing programs to generate visual contents. Our goal is to cover the gap between current needs in industry and what are taught in related courses such as computer vision, computer graphics, and machine learning. This is self-contained class but good knowledge of algorithms, linear algebra, and probability theory are required.
After covering the fundamentals for computer vision and deep learning, we will emphasize generative models from GAN, Auto-regressive models, VAE, Diffusion Models, and Normalizing Flows. We emphasize machine learning basics and applications in image/animation/video generation.
Prereqs:
Basic knowledge of probability and linear algebra; data structures, algorithms; programming experience. Previous experience with image processing will be useful but is not assumed.
Assignments will consist largely of Pytorch or Matlab programming problems. There will be a warm-up assignment to get familiar with basic Pytorch/Matlab commands. We will recommend useful functions to check out per assignment. However, students are expected to practice and pick up Pytorch on their own in order to complete the assignments. The instructor and TAs are happy to help with Pytorch issues during office hours.
If you are unsure if your background is a good match for this course, please come talk to the instructor.
Textbooks:
The main textbook we will use is the following:
This book covers GAN, VAE, and Diffusion Models we teach in the lectures. It is can be downloaded online.
The following books are also useful.
It is covers basics of image filtering, which we teach for the first few lectures. It also covers texture synthesis.
This book has good material on GAN.
This book covers coding on training genartive models using Python and TensorFlow.
Another book that covers coding in TensorFlow.
Assignments: Assignments will be given approximately every two to four weeks (to accomodate midterm and final term preparations). The programming problems will provide hands-on experience working with techniques covered in or related to the lectures. All code and written responses must be completed individually. Most assignments will take significant time to complete. Please start early, and use Ed and/or see us during office hours for help if needed. Please follow instructions in each assignment carefully regarding what to submit and how to submit it.
Extension policy: If you turn in your assignment late, expect points to be deducted. Extensions will be considered on a case-by-case basis, but in most cases they will not be granted. The greater the advance notice of a need for an extension, the greater the likelihood of leniency. For programming assignments, by default, 10 points (out of 100) will be deducted for lateness for each day late. We will use the submission program timestamp to determine time of submission. One day late = from 1 minute to 24 hours past the deadline. Two days late = from 24 hours and 1 minute to 48 hours past the deadline. We will not accept assignments more than 4 days late, or once solutions have been discussed in class, whichever is sooner.
Exams: There is an in-class midterm and an in-class final exam. Both exams will be offered at the listed time only. Neither exam will be offered at a different time to accommodate personal travel plans, internship start dates, interviews, etc.
Participation/attendance: Regular attendance is expected. If for whatever reason you are absent, it is your responsibility to find out what you missed that day. Note that attendance does factor into the final grade. (See Section II of the UTCS Code of Conduct regarding attendance expectations.)
General responsibilities: Beyond the above, your responsibilities in the class are:
Please note the following important dates and deadlines.
Assignments are due about every two or four weeks. The assignment deadlines above are tentative and are provided to help your planning. They are subject to minor shifts if the lecture plan needs to be adjusted slightly according to our pace in class.