CS394R: Reinforcement Learning: Theory and Practice -- Fall 2007: Assignments Page

Assignments for Reinforcement Learning: Theory and Practice

Week 0 (8/30): Class Overview

  • For each class (after the first), be sure to submit a question or comment about the reading by 10pm on the day before class as an email in plain ascii text. I prefer that is be sent in the body of the email, rather than as an attachment. Please use the subject line "class readings for [due date]" and send to Peter and Shivaram (pstone@cs and shivaram@cs). Please include your name in the response. And if you refer explicitly to the reading, please include page numbers.

    • Week 1 (9/4,9/6): Introduction

  • Chapter 1 of the textbook

    • Week 2 (9/11,9/13): Evaluative Feedback

  • Chapter 2 of the textbook

    • Week 3 (9/18,20): The Reinforcement Learning Problem

  • Chapter 3 of the textbook

    • Week 4 (9/25,27): Dynamic Programming

  • Chapter 4 of the textbook

    • Week 5 (10/2,4): Monte Carlo Methods

  • Chapter 5 of the textbook

    • Week 6 (10/9,11): Temporal Difference Learning

  • Chapter 6 of the textbook

    • Week 7 (10/16,18): Eligibility Traces

  • Chapter 7 of the textbook

    • Week 8 (10/23,25): Generalization and Function Approximation

  • Chapter 8 of the textbook
  • Class project proposal due at 12:30pm on Thursday. Please send an email with subject "Project Proposal" with a proposed topic for your class project. I anticipate projects taking one of three forms.
  • Practice (preferred): An implemenation of RL in some domain of your choice - ideally one that you are using for research or in some other class. In this case, please describe the domain and your initial plans on how you intend to implement learning. What will the states and actions be? What algorithm(s) do you expect will be most effective?
  • Theory: A proposal, implementation and testing of an algorithmic modification to an RL algorithm presented in the book. In this case, please describe the modification you propose to investigate and on what type of domain (possibly a toy domain) it is likely to show an improvement over things considered in the book.
  • Mock RL Competition: A program that could participate in the upcoming RL competition, meaning that it uses RL glue and is able to learn on the range of domains that are planned to be used as a part of the competition.

    • Week 9 (10/30,11,1): Planning and Learning

  • Chapter 9 of the textbook

    • Week 10 (11/6,8): Case Studies

  • Chapters 10 and 11 of the textbook

    • Week 11 (11/13,15): Abstraction: Options and Hierarchy

  • Between MDPs and semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning.
    Sutton, R.S., Precup, D., Singh, S.
    Artificial Intelligence 112:181-211, 1999.
    Due Tuesday.
  • The MAXQ Method for Hierarchical Reinforcement Learning.
    Thomas G. Dietterich
    Proceedings of the 15th International Conference on Machine Learning, 1998.
    Due Thursday.

    • Week 12 (11/20): Helicopter Control and Robot Soccer

  • Autonomous helicopter flight via reinforcement learning.
    Andrew Ng, H. Jin Kim, Michael Jordan and Shankar Sastry.
    In S. Thrun, L. Saul, and B. Schoelkopf (Eds.), Advances in Neural Information Processing Systems (NIPS) 17, 2004.
    Due Tuesday.
  • Making a Robot Learn to Play Soccer Using Reward and Punishment.
    Heiko Müller, Martin Lauer, Roland Hafner, Sascha Lange, Artur Merke and Martin Riedmiller.
    Due Tuesday.
    • Note that both papers are due on Tuesday!

    Week 13 (11/27,29): Adaptive Representations and Transfer Learning

  • Evolutionary Function Approximation for Reinforcement Learning.
    Shimon Whiteson and Peter Stone.
    Journal of Machine Learning Research, 7(May):877-917, 2006.
    Due Tuesday.
  • Value Functions for RL-Based Behavior Transfer: A Comparative Study.
    Matthew Taylor, Peter Stone and Yaxin Liu.
    In Proceedings of the Twentieth National Conference on Artificial Intelligence, July 2005.
    Due Thursday.

    • Week 14 (12/4,6): Advice and Multiagent Reinforcement Learning

  • Creating Advice-Taking Reinforcement Learners.
    Richard Maclin and Jude Shavlik.
    Machine Learning, 22, pp. 251-281, 1996.
    Due Tuesday.
  • Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents.
    Ming Tan.
    In Proceedings of the Tenth International Conference on Machine Learning (ICML-93), pages 330-337, 1993.
    Due Thursday.

    • Final Project: due at 12:30pm on Thursday, 12/6

    [Back to Department Homepage]

    Page maintained by Peter Stone
    Questions? Send me mail