CS394R/ECE381V: Reinforcement Learning: Theory and Practice -- Spring 2024: Assignments Page

Assignments for Reinforcement Learning: Theory and Practice

Readings Responses

Each week, there will be a reading response submitted via Canvas. Typically, reading responses will be due 2pm on Monday so that we can discuss the responses in class on Tuesday. If you refer explicitly to the reading, please include quotations and page numbers. Details on expectations for reading responses are on the main class page. Example successful responses from a previous class are available on the sample responses page.

Things to do ASAP

Check that you have access to the class discussion and first reading response on Canvas. Register for the class on edX Edge (see class main page).

Week 1 (1/16): Class Overview, Intro, and Multi-armed Bandits

  • By the day before the first class period, read Chapter 1 of the course textbook (2nd edition) -- Section 1.7 is optional.
  • By Wednesday at 5pm CST, read and submit a reading response for:
  • Post something on the class discussion forum. Posting and/or discussing your reading responses, or introducing yourself is a fine way to do this.
  • Finish Chapter 2 Homework on edx by Sunday at 11:59 PM (please note that every future Homework will be due Friday)

    • Week 2 (1/23): MDPs and Dynamic Programming

    Jump to the resources page.

  • Chapters 3 and 4 of the textbook
  • Complete the reading response on Canvas by Monday at 2pm CST.
  • Complete Homework for Chapters 3+4 on edx by Friday at 11:59 PM CST

    • Week 3 (1/30): Monte Carlo Methods and Temporal Difference Learning

    Jump to the resources page.

  • Chapters 5 and 6 of the textbook
  • Complete the reading response on Canvas by Monday at 2pm CST.
  • Complete Homework for Chapters 5+6 on edx by Friday at 11:59 PM CST
  • Complete Programming Assignment for Chapter 2 on edx by Sunday at 11:59 PM CST

    • Week 4 (2/6): n-step Bootstrapping and Planning

    Jump to the resources page.

  • Chapters 7 and 8 of the textbook. It is OK to skim over 7.4 and 7.6.
  • Complete the reading response on Canvas by Monday at 2pm CST.
  • Complete Homework for Chapters 7+8 on edx by Friday at 11:59 PM CST

    • Week 5 (2/13): On-policy Prediction with Approximation

    Jump to the resources page.

  • Chapter 9 of the textbook
  • Complete the reading response on Canvas by Monday at 2pm CST.
  • Complete Homework for Chapter 9 on edx by Friday 11:59 PM CST

    • Week 6 (2/20): On Policy Control with Approximation and Off Policy Methods with Approximation

    Jump to the resources page.

  • Chapters 10 and Chapter 11 up to the end of Section 11.4 of the textbook. Skim the rest of Chapter 11 if you're interested.
  • Final project proposal due at 11:59pm on Thursday, 3/7.
    See the project page for full details on the project.
  • Complete the reading response on Canvas by Monday at 2pm CST.
  • Complete Homework for Chapters 10+11 on edx by Friday 11:59 PM CST
  • Complete Programming Assignment for Chapters 4,5,6,7 on edx by Sunday at 11:59 PM CST

    • Week 7 (2/27): Eligibility Traces

    Jump to the resources page.

  • Chapter 12 of the textbook. It's OK to skip (or skim) Sections 12.6, 12.9, and 12.11.
  • Complete the mid-semester class survey by 5pm on 3/11. It is available on TBA
  • Final project proposal due at 11:59pm on Thursday, 3/7.
    See the project page for full details on the project.
  • Complete the reading response on Canvas by Monday at 2pm CST.
  • Complete Homework for Chapter 12 on edx by Friday 11:59 PM CST
  • Complete Programming Assignment for Chapters 9+10 on edx by Sunday 11:59 PM CST

    • Week 8 (3/5): Policy Gradient Methods

    Jump to the resources page.

  • Chapter 13 of the textbook.
  • Final project proposal due at 11:59pm on Thursday.
  • Complete the reading response on Canvas by Monday at 2pm CST.
  • Complete Homework for Chapter 13 on edx by Friday 11:59 PM CST
    See the project page for full details on the project.

    • Week 9 (3/19): Applications and Case Studies

    Jump to the resources page.

  • Chapter 16 of the textbook.
  • Complete the reading response on Canvas by Monday at 2pm CST.
  • Midterm exam - Thursday or Friday.
  • Please attempt to complete all programming assignments before the midterm, though the deadline for the final assignment is the week after

    • Week 10 (3/26): Deep RL and the Modern Landscape

    Jump to the resources page.

  • OpenAI's Spinning up in DeepRL pages. Specifically:
  • A Distributional Perspective on Reinforcement Learning by Bellemare, Dabney, and Munos. It's ok to skim Sections 3 and 5.
  • Complete the reading response on Canvas by Monday at 2pm CST.
  • Complete Programming Assignment for Chapters 12+13 on edx by Sunday 11:59 PM CST

    • Week 11 (4/2): Abstraction: Options and Hierarchy

    Jump to the resources page.

  • The Value of Abstraction Ho, M., Abel, D., Griffiths, T., and Littman, M.
  • Chapter 17 of the textbook. You can skim - This is mainly to give you a taste of the things the book hasn't covered.
  • Between MDPs and semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning. Sutton, R.S., Precup, D., Singh, S.
    Artificial Intelligence 112:181-211, 1999. You can skim Sections 5 and 6.
  • Complete the reading response on Canvas by Monday at 2pm CST.

    • Week 12 (4/9): Exploration and Intrinsic Motivation

    Jump to the resources page.

  • Project literature review due at 11:59pm on Thursday.
    See the project page for full details on the project.
  • Bellemare et al.
    Unifying Count-Based Exploration and Intrinsic Motivation
    NIPS 2016.
  • Singh, Lewis, and Barto.
    Where do Rewards Come From?
    2009.
  • Diversity is All You Need: Learning Skills without a Reward Function by Eysenbach, Gupta, Ibarz, and Levine. Read Sections 3 and 4.
  • Complete the reading response on Canvas by Monday at 2pm CST.

    • Week 13 (4/16): Learning from Human Input

    Jump to the resources page.

  • Abbeel, P. and Ng, A.
    Apprenticeship Learning via Inverse Reinforcement Learning
    ICML 2004
  • Knox and Stone
    Combining Manual Feedback with Subsequent MDP Reward Signals for Reinforcement Learning
    AAMAS 2010.
  • Complete the reading response on Canvas by Monday at 2pm CST.

    • Week 14 (4/23): Multiagent RL, Reproducibility, and Wrap-Up

    Jump to the resources page.

  • Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents by Ming Tan
  • Henderson et al., AAAI 2018
    Deep Reinforcement Learning that Matters (p. 1-7 - supplemental material is optional)
  • Complete the reading response on Canvas by Monday at 2pm CST.

    • Final Project (April 29)

  • Due at 11:59pm on Monday, 4/29.
    See the project page for full details on the project.
    •
    [Back to Department Homepage]

    Page maintained by Peter Stone
    Questions? Send me mail