• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
ELDEN: Exploration via Local Dependencies.
Zizhao Wang, Jiaheng
Hu, Peter Stone, and Roberto Martín-Martín.
In Conference on Neural
Information Processing Systems, December 2023.
[PDF]6.2MB [slides.pptx]22.4MB [poster.pdf]856.5kB
Tasks with large state space and sparse rewards present a longstanding challengeto reinforcement learning. In these tasks, an agent needs to explore the statespace efficiently until it finds a reward.To deal with this problem, thecommunity has proposed to augment the reward function with intrinsic reward, abonus signal that encourages the agent to visit interesting states.In thiswork, we propose a new way of defining interesting states for environments withfactored state spaces and complex chained dependencies, where an agent's actionsmay change the value of one entity that, in order, may affect the value ofanother entity. Our insight is that, in these environments, interesting statesfor exploration are states where the agent is uncertain whether (as opposed tohow) entities such as the agent or objects have some influence on each other.Wepresent ELDEN, Exploration via Local DepENdencies, a novel intrinsic reward thatencourages the discovery of new interactions between entities.ELDEN utilizes anovel scheme --- the partial derivative of the learned dynamics to model thelocal dependencies between entities accurately and computationally efficiently.The uncertainty of the predicted dependencies is then used as an intrinsicreward to encourage exploration toward new interactions.We evaluate theperformance of ELDEN on four different domains with complex dependencies,ranging from 2D grid worlds to 3D robotic tasks. In all domains, ELDEN correctlyidentifies local dependencies and learns successful policies, significantly
@InProceedings{jiaheng_hu_neurips2023,
author = {Zizhao Wang and Jiaheng Hu and Peter Stone and Roberto MartÃn-MartÃn},
title = {ELDEN: Exploration via Local Dependencies},
booktitle = {Conference on Neural Information Processing Systems},
year = {2023},
month = {December},
location = {New Orleans, United States},
abstract = {
Tasks with large state space and sparse rewards present a longstanding challenge
to reinforcement learning. In these tasks, an agent needs to explore the state
space efficiently until it finds a reward.
To deal with this problem, the
community has proposed to augment the reward function with intrinsic reward, a
bonus signal that encourages the agent to visit interesting states.
In this
work, we propose a new way of defining interesting states for environments with
factored state spaces and complex chained dependencies, where an agent's actions
may change the value of one entity that, in order, may affect the value of
another entity.
Our insight is that, in these environments, interesting states
for exploration are states where the agent is uncertain whether (as opposed to
how) entities such as the agent or objects have some influence on each other.
We
present ELDEN, Exploration via Local DepENdencies, a novel intrinsic reward that
encourages the discovery of new interactions between entities.
ELDEN utilizes a
novel scheme --- the partial derivative of the learned dynamics to model the
local dependencies between entities accurately and computationally efficiently.
The uncertainty of the predicted dependencies is then used as an intrinsic
reward to encourage exploration toward new interactions.
We evaluate the
performance of ELDEN on four different domains with complex dependencies,
ranging from 2D grid worlds to 3D robotic tasks. In all domains, ELDEN correctly
identifies local dependencies and learns successful policies, significantly
},
}
Generated by bib2html.pl (written by Patrick Riley ) on Tue Nov 19, 2024 10:24:41