• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
Todd Hester and Peter Stone. Intrinsically Motivated Model Learning for a Developing Curious Agent. In The Eleventh International Conference on Development and Learning (ICDL), Nov 2012.
[PDF]147.9kB [postscript]6.9MB
Reinforcement Learning (RL) agents are typically deployed to learn aspecific, concrete task based on a pre-defined reward function.However, in some cases an agent may be able to gainexperience in the domain prior to being given a task. In suchcases, intrinsic motivation can be used to enable the agent to learn auseful model of the environment that is likely to help it learn itseventual tasks more efficiently.This paper presents the \textsctexplore withVariance-And-Novelty-Intrinsic-Rewards algorithm(\textsctexplore-vanir), an intrinsically motivated model-based RLalgorithm. The algorithm learns models of the transition dynamics of adomain using random forests. It calculates two different intrinsicmotivations from this model: one to explore where the model is uncertain,and one to acquire novel experiences that the model has not yet beentrained on. This paper presents experiments demonstrating that thecombination of these two intrinsic rewards enables the algorithm tolearn an accurate model of a domain with no external rewards and thatthe learned model can be used afterward to perform tasks in thedomain. While learning the model, the agent explores the domain in adeveloping and curious way, progressively learning more complexskills. In addition, the experiments show that combining the agent'sintrinsic rewards with external task rewards enables the agent tolearn faster than using external rewards alone.
@InProceedings{ICDL12-hester,
author="Todd Hester and Peter Stone",
title="Intrinsically Motivated Model Learning for a Developing Curious Agent",
booktitle = "The Eleventh International Conference on Development and Learning (ICDL)",
location = "San Diego, CA",
month = "Nov",
year = "2012",
abstract = "Reinforcement Learning (RL) agents are typically deployed to learn a
specific, concrete task based on a pre-defined reward function.
However, in some cases an agent may be able to gain
experience in the domain prior to being given a task. In such
cases, intrinsic motivation can be used to enable the agent to learn a
useful model of the environment that is likely to help it learn its
eventual tasks more efficiently.
This paper presents the \textsc{texplore} with
Variance-And-Novelty-Intrinsic-Rewards algorithm
(\textsc{texplore-vanir}), an intrinsically motivated model-based RL
algorithm. The algorithm learns models of the transition dynamics of a
domain using random forests. It calculates two different intrinsic
motivations from this model: one to explore where the model is uncertain,
and one to acquire novel experiences that the model has not yet been
trained on. This paper presents experiments demonstrating that the
combination of these two intrinsic rewards enables the algorithm to
learn an accurate model of a domain with no external rewards and that
the learned model can be used afterward to perform tasks in the
domain. While learning the model, the agent explores the domain in a
developing and curious way, progressively learning more complex
skills. In addition, the experiments show that combining the agent's
intrinsic rewards with external task rewards enables the agent to
learn faster than using external rewards alone.",
}
Generated by bib2html.pl (written by Patrick Riley ) on Fri Sep 04, 2015 15:05:58