• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
Discovering Creative Behaviors through DUPLEX: Diverse Universal Features for Policy Exploration.
Borja G. Leon,
Francesco Riccio, Kaushik Subramanian, and Peter R. Wurman an Peter Stone.
In
Conference on Neural Information Processing Systems (NeurIPS), December 2024.
Project
website (with videos)
[PDF]1.5MB [slides.pdf]1.4MB [poster.pdf]1.5MB
The ability to approach the same problem from different angles is a cornerstone of human intelligence that leads to robust solutions and effective adaptation to problem variations. In contrast, current RL methodologies tend to lead to policies that settle on a single solution to a given problem, making them brittle to problem variations. Replicating human flexibility in reinforcement learning agents is the challenge that we explore in this work. We tackle this challenge by extending state-of-the-art approaches to introduce DUPLEX, a method that explicitly defines a diversity objective with constraints and makes robust estimates of policies' expected behavior through successor features. The trained agents can (i) learn a diverse set of near-optimal policies in complex highly-dynamic environments and (ii) exhibit competitive and diverse skills in out-of-distribution OOD) contexts. Empirical results indicate that DUPLEX improves over previous methods and successfully learns competitive driving styles in a hyper-realistic simulator (i.e., GranTurismo 7) as well as diverse and effective policies in several multi-context robotics MuJoCo simulations with OOD gravity forces and height limits. To the best of our knowledge, our method is the first to achieve diverse solutions in complex driving simulators and OOD robotic contexts. DUPLEX agents demonstrating diverse behaviors can be found at https: //ai.sony/publications/Discovering-Creative-Behaviors-through- DUPLEX-Diverse-Universal-Features-for-Policy-Exploration/.
@InProceedings{duplex_neurips2024,
author = {Borja G.\ Leon and Francesco Riccio and Kaushik Subramanian and Peter R.\ Wurman an Peter Stone},
title = {Discovering Creative Behaviors through {DUPLEX}: Diverse Universal Features for Policy Exploration},
booktitle = {Conference on Neural Information Processing Systems (NeurIPS)},
year = {2024},
month = {December},
location = {Vancouver, Canada},
abstract = {
The ability to approach the same problem from different
angles is a cornerstone of human intelligence that leads
to robust solutions and effective adaptation to problem
variations. In contrast, current RL methodologies tend
to lead to policies that settle on a single solution to
a given problem, making them brittle to problem
variations. Replicating human flexibility in
reinforcement learning agents is the challenge that we
explore in this work. We tackle this challenge by
extending state-of-the-art approaches to introduce
DUPLEX, a method that explicitly defines a diversity
objective with constraints and makes robust estimates of
policies' expected behavior through successor
features. The trained agents can (i) learn a diverse set
of near-optimal policies in complex highly-dynamic
environments and (ii) exhibit competitive and diverse
skills in out-of-distribution OOD) contexts. Empirical
results indicate that DUPLEX improves over previous
methods and successfully learns competitive driving
styles in a hyper-realistic simulator (i.e., GranTurismo
7) as well as diverse and effective policies in several
multi-context robotics MuJoCo simulations with OOD
gravity forces and height limits. To the best of our
knowledge, our method is the first to achieve diverse
solutions in complex driving simulators and OOD robotic
contexts. DUPLEX agents demonstrating diverse behaviors
can be found at https:
//ai.sony/publications/Discovering-Creative-Behaviors-through-
DUPLEX-Diverse-Universal-Features-for-Policy-Exploration/.
},
wwwnote={<a href="https://ai.sony/publications/Discovering-Creative-Behaviors-through-DUPLEX-Diverse-Universal-Features-for-Policy-Exploration/">Project website</a> (with videos)},
}
Generated by bib2html.pl (written by Patrick Riley ) on Tue Nov 19, 2024 10:24:40