• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learning.
Jiaheng
Hu, Zizhao Wang, Roberto Martín-Martín, and Peter
Stone.
In Conference on Neural Information Parocessing Systems (NeurIPS), December 2024.
A hallmark of intelligent agents is the ability to learn reusable skills purelyfrom unsupervised interaction with the environment. However, existingunsupervised skill discovery methods often learn entangled skills where one skillvariable simultaneously influences many entities in the environment, makingdownstream skill chaining extremely challenging. We propose DisentangledUnsupervised Skill Discovery (DUSDi), a method for learning disentangled skillsthat can be efficiently reused to solve downstream tasks. DUSDi decomposes skillsinto disentangled components, where each skill component only affects one factorof the state space. Importantly, these skill components can be concurrentlycomposed to generate low-level actions, and efficiently chained to tackledownstream tasks through hierarchical Reinforcement Learning. DUSDi defines anovel mutual-information-based objective to enforce disentanglement between theinfluences of different skill components, and utilizes value factorization tooptimize this objective efficiently. Evaluated in a set of challengingenvironments, DUSDi successfully learns disentangled skills, and significantlyoutperforms previous skill discovery methods when it comes to applying thelearned skills to solve downstream tasks. Code and skills visualization atjiahenghu.github.io/DUSDi-site/.
@InProceedings{hu_neurips2024,
author = {Jiaheng Hu and Zizhao Wang and Roberto MartÃn-MartÃn and Peter Stone},
title = {Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learning},
booktitle = {Conference on Neural Information Parocessing Systems (NeurIPS)},
year = {2024},
month = {December},
location = {Vancouver, Canada},
abstract = {A hallmark of intelligent agents is the ability to learn reusable skills purely
from unsupervised interaction with the environment. However, existing
unsupervised skill discovery methods often learn entangled skills where one skill
variable simultaneously influences many entities in the environment, making
downstream skill chaining extremely challenging. We propose Disentangled
Unsupervised Skill Discovery (DUSDi), a method for learning disentangled skills
that can be efficiently reused to solve downstream tasks. DUSDi decomposes skills
into disentangled components, where each skill component only affects one factor
of the state space. Importantly, these skill components can be concurrently
composed to generate low-level actions, and efficiently chained to tackle
downstream tasks through hierarchical Reinforcement Learning. DUSDi defines a
novel mutual-information-based objective to enforce disentanglement between the
influences of different skill components, and utilizes value factorization to
optimize this objective efficiently. Evaluated in a set of challenging
environments, DUSDi successfully learns disentangled skills, and significantly
outperforms previous skill discovery methods when it comes to applying the
learned skills to solve downstream tasks. Code and skills visualization at
jiahenghu.github.io/DUSDi-site/.
},
}
Generated by bib2html.pl (written by Patrick Riley ) on Tue Nov 19, 2024 10:24:40