CS394R: Reinforcement Learning: Theory and Practice -- Fall 2016: Assignments Page

Assignments for Reinforcement Learning: Theory and Practice

Readings Responses

Each week, there will be a reading response submitted via google forms. Be sure to submit a question or comment about each reading by 5pm on Monday in this form. Please include your name at the top in the response. And if you refer explicitly to the reading, please include page numbers. Details on expectations for reading responses are on the main class page. Example successful responses from a previous class are available on the sample responses page.

Things to do ASAP (before the first class if possible)

  • Join the class discussion group by registering for the class on edX (see class main page).

    • Week 1 (1/18): Class Overview, Intro, and Multi-armed Bandits

  • By the day before the first class period at 5pm, read Chapter 1 of the course textbook (2nd edition) -- Section 1.7 is optional.
  • By Wednesday at 5pm, read and submit a reading response for:
  • Post something on the class Piazza. Posting and/or discussing your reading responses is a fine way to do this.
  • Finish Chapter 2 homework on edx by Sunday at 11:59 PM (please note that every non-programming assignment after will be due Friday)

    • Week 2 (1/25): MDPs and Dynamic Programming

    Jump to the resources page.

  • Chapters 3 and 4 of the textbook
  • Post something on the class discussion forum
  • Submit your second reading response by Monday at 5 PM CST
  • You can submit your second reading response here
  • Complete Homework for Chatpers 3+4 on edx by Friday at 11:59 PM CST

    • Week 3 (2/1): Monte Carlo Methods and Temporal Difference Learning

    Jump to the resources page.

  • Chapters 5 and 6 of the textbook
  • Complete Homework for Chapters 5+6 on edx by Friday at 11:59 PM CST
  • Complete Programming Assignment for Chapter 2 on edx by Sunday at 11:59 PM CST
  • You can submit your reading response here

    • Week 4 (2/8): n-step Bootstrapping and Planning

    Jump to the resources page.

  • Chapters 7 and 8 of the textbook. It is OK to skim over 7.4 and 7.6.
  • Complete Homework for Chapters 7+8 on edx by Friday at 11:59 PM CST
  • You can submit your reading response here

    • Week 5 (2/15): On-policy Prediction with Approximation

    Jump to the resources page.

  • Chapter 9 of the textbook
  • Complete Homework for Chapter 9 on edx by Friday 11:59 PM CST
  • You can submit your reading response here

    • Week 6 (2/22): On Policy Control with Approximation and Off Policy Methods with Approximation

    Jump to the resources page.

  • Chapters 10 and Chapter 11 up to the end of Section 11.4 of the textbook. Skim the rest of Chapter 11 if you're interested.
  • Final project proposal due at 11:59pm on Thursday, 3/10.
    See the project page for full details on the project.
  • Complete Homework for Chapters 10+11 on edx by Friday 11:59 PM CST
  • Complete Programming Assignment for Chapters 4,5,6,7 on edx by Sunday at 11:59 PM CST
  • You can submit your reading response here

    • Week 7 (3/1): Eligibility Traces

    Jump to the resources page.

  • Chapter 12 of the textbook. It's OK to skip (or skim) Sections 12.6, 12.9, and 12.11.
  • Complete the mid-semester class survey by 5pm on 3/11. It is available on TBA
  • Final project proposal due at 11:59pm on Thursday, 3/10.
    See the project page for full details on the project.
  • Complete Homework for Chapter 12 on edx by Friday 11:59 PM CST
  • Complete Programming Assignment for Chapters 8+9 on edx by Sunday 11:59 PM CST
  • You can submit your reading response here

    • Week 8 (3/8): Policy Gradient Methods

    Jump to the resources page.

  • Chapter 13 of the textbook.
  • Final project proposal due at 11:59pm on Thursday.
  • Complete Homework for Chapter 13 on edx by Friday 11:59 PM CST
    See the project page for full details on the project.
  • You can submit your reading response here

    • Week 9 (3/22): Applications and Case Studies

    Jump to the resources page.

  • Chapter 16 of the textbook.
  • You can submit your reading response here
  • Midterm exam - Thursday or Friday.
  • Please attempt to complete all programming assignments before the midterm, though the deadline for the final assignment is the week after

    • Week 10 (3/29): Abstraction: Options and Hierarchy

    Jump to the resources page.

  • The Value of Abstraction Ho, M., Abel, D., Griffiths, T., and Littman, M.
  • Between MDPs and semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning. Sutton, R.S., Precup, D., Singh, S.
    Artificial Intelligence 112:181-211, 1999.
  • Complete Programming Assignment for Chapters 12+13 on edx by Sunday 11:59 PM CST
  • You can submit your reading response here

    • Week 11 (4/5): Exploration and Intrinsic Motivation

    Jump to the resources page.

  • Bellemare et al.
    Unifying Count-Based Exploration and Intrinsic Motivation
    NIPS 2016.
  • Singh, Lewis, and Barto.
    Where do Rewards Come From?
    2009.
  • You can submit your reading response here

    • Week 12 (4/12): Learning from Human Input

    Jump to the resources page.

  • Project literature review due at 11:59pm on Thursday.
    See the project page for full details on the project.
  • Abbeel, P. and Ng, A.
    Apprenticeship Learning via Inverse Reinforcement Learning
    ICML 2004
  • Knox and Stone
    Combining Manual Feedback with Subsequent MDP Reward Signals for Reinforcement Learning
    AAMAS 2010.
  • You can submit your reading response here

    • Week 13 (4/19): Multiagent RL and Safe RL

    Jump to the resources page.

  • Michael Littman
    Markov Games as a Framework for Multi-Agent Reinforcement Learning
    ICML, 1994.
  • Michael Bowling and Manuela Veloso
    Rational and Convergent Learning in Stochastic Games
    IJCAI 2001.
  • D.S. Brown and S. Niekum.
    Efficient Probabilistic Performance Bounds for Inverse Reinforcement Learning
    AAAI Conference on Artificial Intelligence, February 2018.
  • You can submit your reading response here

    • Week 14 (4/26): Modern Landscape

    Jump to the resources page.

  • Chapter 17 of the textbook (if you haven't read it already). You can skim - especially the section on options. This is mainly to give you a taste of the things the book hasn't covered.
  • Finn et al.
    Model Agnostic Meta-Learning for Fast Adaptation of Deep Networks
    ICML 2017.
  • Some OpenAI's Spinning up in DeepRL pages. Specifically:
  • You can submit your reading response here

    • Week 15 (5/3): Reproducibility, Evaluation, and Wrap-Up

    Jump to the resources page.

  • Henderson et al., AAAI 2018
    Deep Reinforcement Learning that Matters (p. 1-7 - supplemental material is optional)
  • Agarwal et al., NeurIPS 2021
    Deep Reinforcement Learning at the Edge of the Statistical Precipice
  • You can submit your reading response here

    • Final Project (TBA)

  • Due at 11:59pm on TBA.
    See the project page for full details on the project.
    •
    [Back to Department Homepage]

    Page maintained by Peter Stone
    Questions? Send me mail