Introduction & Logistics

The state of robot manipulation and learning

• An Overview of Dexterous Manipulation. Okamura, Smaby and Cutkosky (ICRA 2000)
• Trends and challenges in robot manipulation. Billard and Kragic (Science 2019)
• A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms. Kroemer, Niekum, Konidaris (JMLR 2019)
• Group discussion

Grasping (Foundations)

• Main: Tutorial on Contact-Based Grasp Synthesis
• Planning Optimal Grasps. C Ferrari and J Canny (ICRA 1992)
• Presenter: Ji Hwan Park

Grasping (Learning)

• Main: Learning: Robotic Grasping of Novel Objects. Ashutosh Saxena, Justin Driemeyer, Justin Kearns, Andrew Y. Ng (NeurIPS 2006)
• Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity. Zhou and Held (CoRL 2023)
• Presenter:

Control (Foundations)

• Main: Tutorial on Robot Control for Manipulation
• Compliance and force control for computer controlled manipulators. Matt Mason (IEEE 1981)
• Presenter: Xinyuan (Diana) Huang

Control (Learning)

• Main: Variable impedance control in end-effector space: An action space for reinforcement learning in contact-rich tasks. Martin-Martin, Roberto, et al. (IROS 2019)
• Learning Variable Impedance Control for Contact Sensitive Taskss. Bogdanovic et al. (RAL 2020)
• Presenter: Jonathan Madera

Motion Planning (Foundations)

• Main: Tutorial on Motion Planning
• Spatial Planning: A Configuration Space Approach. Tomás Lozano-Pérez (IEEE 1980)
• Presenter:

Motion Planning (Learning)

• Main: Motion Policy Networks. Fishman et al. (CoRL 2022)
• Presenter: Carl Stott

Task and Motion Planning (Foundations)

• Main: Tutorial on Planning and TAMP
• Hierarchical Task and Motion Planning in the Now. Leslie Pack Kaelbling, Tomás Lozano-Pérez (ICRA 2011)
• Presenter: Alex Navarro

Task and Motion Planning (Learning)

• Main: Do as i can, not as i say: Grounding language in robotic affordances. Ahn, Michael, et al. (CoRL 2022)
• SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning. Rana et al. (CoRL 2023)
• Presenter: Arpit Bahety

Imitation Learning for Manipulation (Foundations)

• Main: Tutorial on Imitation learning
• Is imitation learning the route to humanoid robots?. Schaal, Stefan (Trends in cognitive sciences 1999)
• Presenter: Fernando Palafox

Imitation Learning for Manipulation I

• Main: Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. . Zhao et al. (2023)
• Diffusion policy: Visuomotor policy learning via action diffusion.. Chi C, Feng S, Du Y, Xu Z, Cousineau E, Burchfiel B, Song S. (2023)
• Presenter: Xingru Zhou

Imitation Learning for Manipulation II

• Main: Rt-1: Robotics transformer for real-world control at scale. Brohan, Anthony, et al. (2022)
• Open X-Embodiment: Robotic Learning Datasets and RT-X Models. Open-X Embodiment Collaboration (2023)
• Presenter: Heath Boyea

Reinforcement Learning for Manipulation (Foundations)

• Main: Tutorial on Reinforcement Learning
• Proximal policy optimization algorithms. Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O. (Arxiv 2017)
• Presenter: Crasun Jans

Reinforcement Learning for Manipulation (Model Free)

• End-to-End Training of Deep Visuomotor Policies. Sergey Levine*, Chelsea Finn*, Trevor Darrell, Pieter Abbeel (JMLR 2017)
• Main: Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations. Rajeswaran et al. (RSS 2018)
• Presenter: Shah Zuhairul Huda

Reinforcement Learning for Manipulation (Model Based)

• Main: Tutorial Model-based Model-free RL
• Mastering Diverse Domains through World Models. Hafner, D., Pasukonis, J., Ba, J., Lillicrap, T. (2023)
• Presenter: Abayomi Adekanmbi

No class / Spring Break

No class / Spring Break

Intermediate Project Milestone

• Presenter: All teams

Simulators for Robot Manipulation

• Main: Mujoco: A physics engine for model-based control. E Todorov, T Erez, Y Tassa (2017)
• robosuite: A Modular Simulation Framework and Benchmark for Robot Learning. Zhu et al. (2017)
• Presenter: Dong Ho Kang (DK)

Sim2real for Robot Manipulation

• Main: In-hand object rotation via rapid motor adaptation. Haozhi Qi, Ashish Kumar, Roberto Calandra, Yi Ma, Jitendra Malik (CoRL 2017)
• TWIST: Teacher-Student World Model Distillation for Efficient Sim-to-Real Transfer. Jun Yamada, Marc Rigter, Jack Collins, Ingmar Posner (2023)
• Presenter: Max Rudolph

Benchmarking Robot Manipulation

• Main: Behavior-1k: A benchmark for embodied ai with 1,000 everyday activities and realistic simulation. Li, Chengshu, et al. (CoRL 2022)
• Rlbench: The robot learning benchmark & learning environment. James S, Ma Z, Arrojo DR, Davison AJ. (RAL 2020)
• Presenter: Caleb Horan

Language in Robot Manipulation (Foundations)

• Main: Tutorial on Deep NLP
• Attention is all you need. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (NeurIPS 2017)
• Tutorial on Transformers (video)
• Presenter: Eric Qian

Language in Robot Manipulation (Learning)

• Main: MUTEX: Learning Unified Policies from Multimodal Task Specifications. Shah R, Martín-Martín R, Zhu Y (CoRL 2023)
• No, to the Right: Online Language Corrections for Robotic Manipulation via Shared Autonomy.. Cui Y, Karamcheti S, Palleti R, Shivakumar N, Liang P, Sadigh D. (HRI 2023)
• Presenter: S P Sharan

Robot Manipulation Systems in the Real World (Foundations)

• Tutorial on Robotic Systems. (2017)
• Main: Lessons from the amazon picking challenge: Four aspects of building robotic systems. Eppner C, Höfer S, Jonschkowski R, Martín-Martín R, Sieverling A, Wall V, Brock O (RSS 2015)
• Presenter: Evan Rosenthal

Robot Manipulation Systems in the Real World (Learning)

• Main: The Ingredients of Real-World Robotic Reinforcement Learning. Henry Zhu, Justin Yu, Abhishek Gupta, Dhruv Shah, Kristian Hartikainen, Avi Singh, Vikash Kumar, Sergey Levine (2020)
• Presenter: Walter Stark

Guest Lecture Dr. Chris Paxton (FAIR)

Guest Lecture Prof. Beomjoon Kim (KAIST)

Spotlight Spotlight Presentations + Poster Session I

Spotlight Spotlight Presentations + Poster Session II