CS391R: Robot Learning

Syllabus

Schedule

All deadlines are 9:59pm CT. Paper reviews / abstracts are due by the previous night of the presentation date.
Readings: ● required ○ supporting material and additional readings

Date	Topic	Abstract Submission
Week 1 Tue, Aug 23	Lecture Introduction & Logistics (by Roberto) Lecture Slide [To get you inspired] What is a robot?. IEEE-Robotics and Automation Society (RAS)
Part I: Robot Perception
Week 1 Thu, Aug 25	Lecture Overview of Robot Perception (by Roberto) Lecture Slide The Limits and Potentials of Deep Learning for Robotics. Niko Sünderhauf, Oliver Brock, Walter Scheirer, Raia Hadsell, Dieter Fox, Jürgen Leitner, Ben Upcroft, Pieter Abbeel, Wolfram Burgard, Michael Milford, Peter Corke (2018)
Week 2 Tue, Aug 30	2D Vision ImageNet Classification with Deep Convolutional Neural Networks. KAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton (2012) Presenter: Shuozhe Li Slides You Only Look Once: Unified, Real-Time Object Detection. Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi (2015) Presenter: Taijing Chen Slides Mask R-CNN. Kaiming He, Georgia Gkioxari, Piotr Dollar, Ross Girshick (2017) Presenter: Zichao Hu Slides Lecture from CS231n, Stanford University. Fei-Fei Li and Jiajun Wu	Required
Week 2 Thu, Sept 1	2D Vision in Robotics Dex-Net 1.0: A cloud-based network of 3D objects for robust grasp planning using a Multi-Armed Bandit model with correlated rewards. Jeff Mahler, Florian Pokorny, Brian Hou, Melrose Roderick, Michael Laskey, Mathieu Aubry, Kai Kohlhoff, Torsten Kroeger, James Kuffner, Ken Goldberg (2017) Presenter: N/A Learning to Look Around: Intelligently Exploring Unseen Environments for Unknown Tasks. Dinesh Jayaraman, Kristen Grauman (2017) Presenter: Jeff Hu Slides Embodied Amodal Recognition: Learning to Move to Perceive Objects. Jianwei Yang, Zhile Ren, Mingze Xu, Xinlei Chen, David J. Crandall, Devi Parikh, Dhruv Batra (2019) Presenter: Annabel To Slides	Required
Week TBD	Tutorial Simulator Tutorial (by Yifeng Zhu)
Week 3 Tue, Sept 6	3D Vision PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space (2). Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas (2017) Presenter: Jiaxin Lu Slides DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion. Chen Wang, Danfei Xu, Yuke Zhu, Roberto Martín-Martín, Cewu Lu, Li Fei-Fei, Silvio Savarese (2019) Presenter: N/A NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng (2020) Presenter: Elvin Yang Slides Lecture from CS231n, Stanford University. Jiajun Wu	Required
Week 3 Thu, Sept 8	3D Vision in Robotics Tracking People by Predicting 3D Appearance, Location and Pose (2). Jathushan Rajasegaran, Georgios Pavlakos, Angjoo Kanazawa, Jitendra Malik (2022) Presenter: Rafael Seroussi Slides DART: Dense Articulated Real-Time Tracking. Tanner Schmidt, Richard Newcombe, Dieter Fox (2010) Presenter: N/A Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representations. Zhenyu Jiang, Yifeng Zhu, Maxwell Svetlik, Kuan Fang, Yuke Zhu (2021) Presenter: Cheng-Chun Hsu Slides	Required
Week 4 Tue, Sept 13	Transformer Attention Attention is All You Need. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin (2017) Presenter: Alex Chandler Slides CLIP: Learning Transferable Visual Models From Natural Language Supervision. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, Ilya Sutskever (2021) Presenter: N/A VideoGPT: Video Generation using VQ-VAE and Transformers. Wilson Yan, Yunzhi Zhang, P. Abbeel, A. Srinivas (2021) Presenter: Om Pandey Slides	Required
Week 4 Thu, Sept 15	Recursive State Estimation Differentiable Particle Filters: End-to-End Learning with Algorithmic Priors. Rico Jonschkowski, Divyam Rastogi, Oliver Brock (2018) Presenter: Frank Regal Slides Online Interactive Perception of Articulated Objects with Multi-Level Recursive Estimation Based on Task-Specific Priors. Roberto Martín-Martín, Oliver Brock (2014) Presenter: Gu-Choel Jeong Slides Multimodal sensor fusion with differentiable filters. Michelle A Lee, Brent Yi, Roberto Martín-Martín, Silvio Savarese, Jeannette Bohg (2020) Presenter: Steven Shi Slides A Brief Tutorial On Recursive Estimation With Examples From Intelligent Vehicle Applications. Hao Li	Required
Project Proposal Deadline : Thu, Sept 15
Week 5 Tue, Sept 20	Interactive Perception Active Perception. Ruzena Bajcsy (1988) Presenter: Jahnavi Malagavalli Slides Interactive Perception: Leveraging Action in Perception and Perception in Action. Jeannette Bohg, Karol Hausman, Bharath Sankaran, Oliver Brock, Danica Kragic, Stefan Schaal, Gaurav Sukhatme (2016) Presenter: N/A The Curious Robot: Learning Visual Representations via Physical Interactions. Lerrel Pinto, Dhiraj Gandhi, Yuanfeng Han, Yong-Lae Park, Abhinav Gupta (2016) Presenter: Matthew Healy Slides	Required
Part II: Robot Decision Making
Week 5 Thu, Sept 22	Lecture Overview of Robot Decision Making
Week 6 Tue, Sept 27	Model-Free Reinforcement Learning Trust Region Policy Optimization. John Schulman, Sergey Levine, Philipp Moritz, Michael I. Jordan, Pieter Abbeel (2015) Presenter: Abayomi Adekanmbi Slides Soft Actor-Critic Algorithms and Applications. Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, Sergey Levine (2018) Presenter: Yasasvi Peruvemba Slides Learning Dexterous In-Hand Manipulation. OpenAI: Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, Bob McGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, Jonas Schneider, Szymon Sidor, Josh Tobin, Peter Welinder, Lilian Weng, Wojciech Zaremba(2019) Presenter: Franke Tang Slides	Required
Week 6 Thu, Sept 29	Model-Free Reinforcement Learning and Meta-Learning Proximal Policy Optimization Algorithms . John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov (2017) Presenter: Roberto Ruiz Slides Continuous Control with Deep Reinforcement Learning. Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, Daan Wierstra (2016) Presenter: Jeff Bonyun Slides Model-agnostic meta-learning for fast adaptation of deep networks. Chelsea Finn, Pieter Abbeel, Sergey Levine (2017) Presenter: Lamha Goel Slides	Required
Week 7 Tue, Oct 4	Model-Based Reinforcement Learning End-to-end training of deep visuomotor policies. Sergey Levine, Chelsea Finn, Trevor Darrell, Pieter Abbeel (2016) Presenter: Xixi Hu Slides Deep Visual Foresight for Planning Robot Motion . Chelsea Finn, Sergey Levine (2017) Presenter: N/A Dream to Control: Learning Behaviors by Latent Imagination. Danijar Hafner, Timothy Lillicrap, Jimmy Ba, Mohammad Norouzi (2019) Presenter: Nathaniel Plaxton Slides	Required
Week 7 Thu, Oct 6	Batch (Offline) Reinforcement Learning Off-Policy Deep Reinforcement Learning without Exploration . Scott Fujimoto, David Meger, Doina Precup (2019) Presenter: Yian Wong Slides Conservative q-learning for offline reinforcement learning. Aviral Kumar, Aurick Zhou, George Tucker, Sergey Levine (2020) Presenter: Siddhant Agarwal Slides Iris: Implicit reinforcement without interaction at scale for learning control from offline robot manipulation data. Ajay Mandlekar, Fabio Ramos, Byron Boots, Silvio Savarese, Li Fei-Fei, Animesh Garg, Dieter Fox (2019) Presenter: Srinath Tankasala Slides Tutorial on Offline Reinforcement Learning, NeurIPS workshop. Aviral Kumar, Sergey Levine	Required
Week 8 Tue, Oct 11	Imitation as Supervised Learning Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation . Tianhao Zhang, Zoe McCarthy, Owen Jow, Dennis Lee, Xi Chen, Ken Goldberg, Pieter Abbeel (2017) Presenter: Muhammad Muaz Slides A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. Stephane Ross, Geoffrey J. Gordon, J. Andrew Bagnell (2010) Presenter: Christina Petlowany Slides Learning Latent Plans from Play. Corey Lynch, Mohi Khansari, Ted Xiao, Vikash Kumar, Jonathan Tompson, Sergey Levine, Pierre Sermanet (2019) Presenter: Ruchira Ray Slides Is imitation learning the route to humanoid robots?. Stefan Schaal	Required
Week 8 Thu, Oct 13	Inverse Reinforcement Learning Apprenticeship Learning via Inverse Reinforcement Learning . Pieter Abbeel, Andrew Ng (2004) Presenter: Anshul Tomar Slides Maximum Entropy Inverse Reinforcement Learning. Brian D. Ziebart, Andrew Maas, J.Andrew Bagnell, Anind K. Dey (2008) Presenter: Tyler Ingebrand Slides Cooperative Inverse Reinforcement Learning. Dylan Hadfield-Menell, Stuart J. Russell, Pieter Abbeel, Anca Dragan (2016) Presenter: Davin Lawrence Slides	Required
Week 9 Tue, Oct 18	Human-in-the-Loop Learning Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spaces . Garrett Warnell, Nicholas Waytowich, Vernon Lawhern, Peter Stone (2018) Presenter: Jordi Ramos Slides HG-DAgger: Interactive Imitation Learning with Human Experts. Michael Kelly, Chelsea Sidrane, Katherine Driggs-Campbell, Mykel J. Kochenderfer (2019) Presenter: Adeet Parikh Slides Human-in-the-Loop Imitation Learning using Remote Teleoperation. Ajay Mandlekar, Danfei Xu, Roberto Martín-Martín, Yuke Zhu, Li Fei-Fei, Silvio Savarese (2020) Presenter: Rutav Shah Slides	Required
Week 9 Thu, Oct 20	Temporal Abstraction 1: Hierarchical Policy Learning The Option-Critic Architecture . Pierre-Luc Bacon, Jean Harb, Doina Precup (2017) Presenter: Kyle Soares Slides Robot Learning from Demonstration by Constructing Skill Trees. George Konidaris, Scott Kuindersma (2012) Presenter: Eric Hsiung Slides FeUdal Networks for Hierarchical Reinforcement Learning. Alexander Sasha Vezhnevets, Simon Osindero, Tom Schaul, Nicolas Heess, Max Jaderberg, David Silver, Koray Kavukcuoglu (2017) Presenter: Aashish Gottipati Slides	Required
Project Milestone Deadline : Thu, Oct 20
Week 10 Tue, Oct 25	Temporal abstraction 2: Task and Motion Planning Hierarchical Task and Motion Planning in the Now . Leslie Pack Kaelbling and Tomas Lozáno-Pérez (2021) Presenter: Andrew Wu Slides Integrating Task-Motion Planning with Reinforcement Learning for Robust Decision Making in Mobile Robots. Yuqian Jiang, Fangkai Yang, Shiqi Zhang, Peter Stone (2018) Presenter: Mihir Suvarna Slides Learning Neuro-Symbolic Skills for Bilevel Planning. Tom Silver, Ashay Athalye, Joshua B. Tenenbaum, Tomas Lozano-Perez, Leslie Pack Kaelbling (2022) Presenter: Noah Patton Slides	Required
Week 10 Thu, Oct 27	Language in Robotics Structformer: Learning spatial structure for language-guided semantic rearrangement of novel objects . Weiyu Liu, Chris Paxton, Tucker Hermans, Dieter Fox (2021) Presenter: Sharachchandra Bhat Slides Concept2robot: Learning manipulation concepts from instructions and human demonstrations. Lin Shao, Toki Migimatsu, Qiang Zhang, Karen Yang, and Jeannette Bohg (2020) Presenter: Albert Yu Slides Using Natural Language for Reward Shaping in Reinforcement Learning. Prasoon Goyal, Scott Niekum, Ray Mooney (2019) Presenter: Urvi Shah Slides	Required
Part III: From Theory To Robotic Systems
Week 11 Tue, Nov 1	Challenges in Robot Learning Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning . Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Avnish Narayan, Hayden Shively, Adithya Bellathur, Karol Hausman, Chelsea Finn, Sergey Levine (2019) Presenter: Jake Grigsby Slides BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments. Sanjana Srivastava, Chengshu Li, Michael Lingelbach, Roberto Martín-Martín, Fei Xia, Kent Vainio, Zheng Lian, Cem Gokmen, Shyamal Buch, C. Karen Liu, Silvio Savarese, Hyowon Gweon, Jiajun Wu, Li Fei-Fei (2021) Presenter: Jane Yang Slides Rlbench: The robot learning benchmark & learning environment. Stephen James, Zicong Ma, David Rovick Arrojo, Andrew J. Davison (2019) Presenter: Arjun Somayazulu Slides	Required
Week 11 Thu, Nov 3	Building Robotic Systems Real-time Perception meets Reactive Motion Generation . Daniel Kappler, Franziska Meier, Jan Issac, Jim Mainprice, Cristina Garcia Cifuentes, Manuel Wüthrich, Vincent Berenz, Stefan Schaal, Nathan Ratliff, Jeannette Bohg (2017) Presenter: Omeed Tehrani Slides Four aspects of building robotic systems: lessons from the Amazon Picking Challenge. Clemens Eppner, Sebastian Höfer, Rico Jonschkowski, Roberto Martín-Martín, Arne Sieverling, Vincent Wall, Oliver Brock (2012) Presenter: Yashvi Shah Slides Learning Agile and Dynamic Motor Skills for Legged Robots. Jemin Hwangbo, Joonho Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun, Marco Hutter (2019) Presenter: Jahnavi Malagavalli Slides	Required
Week 12 Tue, Nov 8	Sim-to-Real Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects . Jonathan Tremblay, Thang To, Balakumar Sundaralingam, Yu Xiang, Dieter Fox, Stan Birchfield (2018) Presenter: Kunjal Shah Slides Driving Policy Transfer via Modularity and Abstraction. Matthias Müller, Alexey Dosovitskiy, Bernard Ghanem, Vladlen Koltun (2018) Presenter: Yuhan Gao Slides Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks. Stephen James, Paul Wohlhart, Mrinal Kalakrishnan, Dmitry Kalashnikov, Alex Irpan, Julian Ibarz, Sergey Levine, Raia Hadsell, Konstantinos Bousmalis (2019) Presenter: Haoyue Cui Slides	Required
Week 12 Thu, Nov 10	Lecture Guest Lecture Language Model as Decision Makers (concrete title TBD). Fei Xia (Google)
Week 13 Tue, Nov 15	Lecture Guest Lecture TBD. TBD
Week 13 Thu, Nov 17	Lecture Ajay Mandlekar TBD. TBD
Week 14 Tue, Nov 22	Holiday No Class / Thanksgiving
Week 14 Thu, Nov 24	Holiday No Class / Thanksgiving
Week 15 Tue, Nov 29	Spotlight Spotlight Presentations + Poster Session I
Week 15 Thu, Dec 1	Spotlight Spotlight Presentations + Poster Session II
Project Final Report Deadline : Fri, Dec 9