CS394R: Reinforcement Learning: Theory and Practice -- Fall 2016: Assignments Page

Assignments for Reinforcement Learning: Theory and Practice

Readings Responses

Each week, there will be a reading response section on the edX course page under the Reading Responses tab. Be sure to submit a question or comment about each reading by 5pm on Monday in this section. Please include your name at the top in the response. And if you refer explicitly to the reading, please include page numbers. Details on expectations for reading responses are on the main class page. Example successful responses from a previous class are available on the sample responses page.

Things to do ASAP (before the first class if possible)

  • Join the class discussion group by registering for the class on edX Edge (see class main page).

    • Week 0 (8/29): Class Overview

  • By the day before the first class period at 5pm, read and react to (see the class main page) Chapter 1 of the course textbook (2nd edition) -- Section 1.7 is optional.

    • Week 1 (9/3): Multi-armed Bandits and MDPs

    Jump to the resources page.

  • Introduction to Part I of the textbook (just one page)
  • Chapter 2
  • Chapter 3
  • Post something on the edX discussion forum
  • Each week, there will be a reading response section on the edX course page under the Reading Responses tab. Be sure to submit a question or comment about each reading by 5pm on Monday in this section. Please include your name at the top in the response. And if you refer explicitly to the reading, please include page numbers. Details on expectations for reading responses are on the main class page. Example successful responses from a previous class are available on the sample responses page.

    • Week 2 (9/10): Dynamic Programming and Monte Carlo Methods

    Jump to the resources page.

  • Chapters 4 and 5 of the textbook (2nd edition)
  • Do edX exercises and programming assignments

    • Week 3 (9/17): TD Learning and n-step Bootstrapping

    Jump to the resources page.

  • Chapters 6 and 7 of the textbook. It is OK to skim over 7.4 and 7.6.

    • Week 4 (9/24): Planning and Learning with Tabular Methods

    Jump to the resources page.

  • Chapter 8 of the textbook

    • Week 5 (10/1): On-policy Prediction with Approximation

    Jump to the resources page.

  • Chapter 9 of the textbook

    • Week 6 (10/8): On Policy Control with Approximation and Off Policy Methods with Approximation

    Jump to the resources page.

  • Chapters 10 and Chapter 11 up to the end of Section 11.4 of the textbook. Skim the rest of Chapter 11 if you're interested.
  • Final project proposal due at 11:59pm on Thursday, 10/24.
    See the project page for full details on the project.
  • Complete the mid-semester class survey by 2pm on 10/15. It is available on Canvas.
    If you are enrolled in the CS section, you can use this link.
    If you are enrolled in the ECE section, you can use this link.

    • Week 7 (10/15): Eligibility Traces

    Jump to the resources page.

  • Chapter 12 of the textbook. It's OK to skip (or skim) Sections 12.6, 12.9, and 12.11.
  • Final project proposal due at 11:59pm on Thursday, 10/24.
    See the project page for full details on the project.

    • Week 8 (10/22): Policy Gradient Methods

    Jump to the resources page.

  • Chapter 13 of the textbook.
  • Final project proposal due at 11:59pm on Thursday.
    See the project page for full details on the project.

    • Week 9 (10/29): Applications and Case Studies

    Jump to the resources page.

  • Chapter 16 of the textbook.
  • Midterm exam - Thursday or Friday.

    • Week 10 (11/5): Abstraction: Options and Hierarchy

    Jump to the resources page.

  • The Value of Abstraction Ho, M., Abel, D., Griffiths, T., and Littman, M.
  • Between MDPs and semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning. Sutton, R.S., Precup, D., Singh, S.
    Artificial Intelligence 112:181-211, 1999.
    •

    Week 11 (11/12): Learning from Human Input

    Jump to the resources page.

  • Project literature review due at 11:59pm on Thursday.
    See the project page for full details on the project.
  • Abbeel, P. and Ng, A.
    Apprenticeship Learning via Inverse Reinforcement Learning
    ICML 2004
  • Knox and Stone
    Combining Manual Feedback with Subsequent MDP Reward Signals for Reinforcement Learning
    AAMAS 2010.

    • Week 12 (11/19): Multiagent RL and Safe RL

    Jump to the resources page.

  • Michael Littman
    Markov Games as a Framework for Multi-Agent Reinforcement Learning
    ICML, 1994.
  • Michael Bowling and Manuela Veloso
    Rational and Convergent Learning in Stochastic Games
    IJCAI 2001.
  • D.S. Brown and S. Niekum.
    Efficient Probabilistic Performance Bounds for Inverse Reinforcement Learning
    AAAI Conference on Artificial Intelligence, February 2018.

    • Week 13 (11/26): Exploration and Intrinsic Motivation

    Jump to the resources page.

  • Bellemare et al.
    Unifying Count-Based Exploration and Intrinsic Motivation
    NIPS 2016.

    • Week 14 (12/3): Modern Landscape

    Jump to the resources page.

  • Chapter 17 of the textbook. You can skim - especially the section on options. This is mainly to give you a taste of the things the book hasn't covered.
  • Finn et al.
    Model Agnostic Meta-Learning for Fast Adaptation of Deep Networks
    ICML 2017.
  • Haarnoja et al.
    Soft Actor-Critic:Off-Policy Maximum Entropy Deep ReinforcementLearning with a Stochastic Actor
    ICML 2018.
    project website

    • Final Project (12/9)

  • Due at 11:59pm on Monday, 12/9.
    See the project page for full details on the project.
    •
    [Back to Department Homepage]

    Page maintained by Peter Stone
    Questions? Send me mail