• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
SkiLD: Unsupervised Skill Discovery Guided by Factor Interactions.
Zizhao
Wang, Jiaheng Hu, Caleb Chuck, Stephen
Chen, Roberto Martín-Martín, Amy Zhang, Scott Niekum, and Peter
Stone.
In Conference on Neural Information Processing Systems (NeurIPS), December 2024.
Unsupervised skill discovery carries the promise that an intelligent agent canlearn reusable skills through autonomous, reward-free environment interaction.Existing unsupervised skill discovery methods learn skills by encouragingdistinguishable behaviors that cover diverse states. However, in complexenvironments with many state factors (e.g., household environments with manyobjects), learning skills that cover all possible states is impossible, andnaively encouraging state diversity often leads to simple skills that are notideal for solving downstream tasks. This work introduces Skill Discovery fromLocal Dependencies (SkiLD), which leverages state factorization as a naturalinductive bias to guide the skill learning process. The key intuition guidingSkiLD is that skills that induce diverse interactions between state factors areoften more valuable for solving downstream tasks. To this end, SkiLD develops anovel skill learning objective that explicitly encourages the mastering of skillsthat effectively induce different interactions within an environment. We evaluateSkiLD in several domains with challenging, long-horizon sparse reward tasksincluding a realistic simulated household robot domain, where SkiLD successfullylearns skills with clear semantic meaning and shows superior performance comparedto existing unsupervised reinforcement learning methods that only maximize statecoverage. Code and visualizations are at https://wangzizhao.github.io/SkiLD/.
@InProceedings{zizhao_neurips2024,
author = {Zizhao Wang and Jiaheng Hu and Caleb Chuck and Stephen Chen and Roberto MartÃn-MartÃn and Amy Zhang and Scott Niekum and Peter Stone},
title = {SkiLD: Unsupervised Skill Discovery Guided by Factor Interactions},
booktitle = {Conference on Neural Information Processing Systems (NeurIPS)},
year = {2024},
month = {December},
location = {Vancouver, Canada},
abstract = {Unsupervised skill discovery carries the promise that an intelligent agent can
learn reusable skills through autonomous, reward-free environment interaction.
Existing unsupervised skill discovery methods learn skills by encouraging
distinguishable behaviors that cover diverse states. However, in complex
environments with many state factors (e.g., household environments with many
objects), learning skills that cover all possible states is impossible, and
naively encouraging state diversity often leads to simple skills that are not
ideal for solving downstream tasks. This work introduces Skill Discovery from
Local Dependencies (SkiLD), which leverages state factorization as a natural
inductive bias to guide the skill learning process. The key intuition guiding
SkiLD is that skills that induce diverse interactions between state factors are
often more valuable for solving downstream tasks. To this end, SkiLD develops a
novel skill learning objective that explicitly encourages the mastering of skills
that effectively induce different interactions within an environment. We evaluate
SkiLD in several domains with challenging, long-horizon sparse reward tasks
including a realistic simulated household robot domain, where SkiLD successfully
learns skills with clear semantic meaning and shows superior performance compared
to existing unsupervised reinforcement learning methods that only maximize state
coverage. Code and visualizations are at https://wangzizhao.github.io/SkiLD/.
},
}
Generated by bib2html.pl (written by Patrick Riley ) on Wed Jan 15, 2025 08:40:53