• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
The EMPATHIC Framework for Task Learning from Implicit Human Feedback.
Yuchen
Cui, Qiping Zhang, Alessandro Allievi, Peter Stone, Scott
Niekum, and W. Bradley Knox.
In Proceedings of the 4th Conference on Robot
Learning (CoRL 2020), November 2020.
5-minute video presentation;
47-minute in-depth talk.
Project website.
Raw
data from experiments.
[PDF]6.4MB [slides.pptx]59.4MB
Reactions such as gestures, facial expressions, and vocalizations are an abundant, naturally occurring channel of information that humans provide during interactions. A robot or other agent could leverage an understanding of such implicit human feedback to improve its task performance at no cost to the human. This approach contrasts with common agent teaching methods based on demonstrations, critiques, or other guidance that need to be attentively and intentionally provided. In this paper, we first define the general problem of learning from implicit human feedback and then propose to address this problem through a novel data-driven framework, EMPATHIC. This two-stage method consists of (1) mapping implicit human feedback to relevant task statistics such as reward, optimality, and advantage; and (2) using such a mapping to learn a task. We instantiate the first stage and three second-stage evaluations of the learned mapping. To do so, we collect a dataset of human facial reactions while participants observe an agent execute a sub-optimal policy for a prescribed training task. We train a deep neural network on this data and demonstrate its ability to (1) infer relative reward ranking of events in the training task from prerecorded human facial reactions; (2) improve the policy of an agent in the training task using live human facial reactions; and (3) transfer to a novel domain in which it evaluates robot manipulation trajectories.
@InProceedings{CORL20-Cui,
author = {Yuchen Cui and Qiping Zhang and Alessandro Allievi and Peter Stone and Scott Niekum and W. Bradley Knox},
title = {The {EMPATHIC} Framework for Task Learning from Implicit Human Feedback},
booktitle = {Proceedings of the 4th Conference on Robot Learning (CoRL 2020)},
location = {Cambridge MA, USA},
month = {November},
year = {2020},
abstract = {
Reactions such as gestures, facial expressions, and vocalizations are an
abundant, naturally occurring channel of information that humans provide
during interactions. A robot or other agent could leverage an understanding
of such implicit human feedback to improve its task performance at no cost
to the human. This approach contrasts with common agent teaching methods
based on demonstrations, critiques, or other guidance that need to be
attentively and intentionally provided. In this paper, we first define the
general problem of learning from implicit human feedback and then propose to
address this problem through a novel data-driven framework, EMPATHIC. This
two-stage method consists of (1) mapping implicit human feedback to relevant
task statistics such as reward, optimality, and advantage; and (2) using such
a mapping to learn a task. We instantiate the first stage and three second-stage
evaluations of the learned mapping. To do so, we collect a dataset of human
facial reactions while participants observe an agent execute a sub-optimal
policy for a prescribed training task. We train a deep neural network on this
data and demonstrate its ability to (1) infer relative reward ranking of events
in the training task from prerecorded human facial reactions; (2) improve the
policy of an agent in the training task using live human facial reactions; and
(3) transfer to a novel domain in which it evaluates robot manipulation
trajectories.
},
wwwnote={<a href="https://www.youtube.com/watch?v=sTvNUpsf4P8">5-minute video presentation</a>;
<a href="https://www.youtube.com/watch?v=7FCttQyl9ag&feature=emb_logo">47-minute in-depth talk</a>.<br>
<a href="https://sites.google.com/utexas.edu/empathic">Project website</a>.<br>
<a href="https://zenodo.org/record/4290896#.X76zmcJMGV6">Raw data from experiments</a>.},
}
Generated by bib2html.pl (written by Patrick Riley ) on Tue Nov 19, 2024 10:24:42