• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
Hierarchical Model-Based Reinforcement Learning: Rmax + MAXQ.
Nicholas
K. Jong and Peter Stone.
In Proceedings of the Twenty-Fifth
International Conference on Machine Learning, July 2008.
ICML 2008
[PDF]157.5kB [postscript]370.2kB
Hierarchical decomposition promises to help scale reinforcement learning algorithms naturally to real-world problems by exploiting their underlying structure. Model-based algorithms, which provided the first finite-time convergence guarantees for reinforcement learning, may also play an important role in coping with the relative scarcity of data in large environments. In this paper, we introduce an algorithm that fully integrates modern hierarchical and model-learning methods in the standard reinforcement learning setting. Our algorithm, \textscR-maxq, inherits the efficient model-based exploration of the \textscR-max algorithm and the opportunities for abstraction provided by the MAXQ framework. We analyze the sample complexity of our algorithm, and our experiments in a standard simulation environment illustrate the advantages of combining hierarchies and models.
@InProceedings{ICML08-jong,
author="Nicholas K.\ Jong and Peter Stone",
title="Hierarchical Model-Based Reinforcement Learning: {Rmax} + {MAXQ}",
booktitle="Proceedings of the Twenty-Fifth International Conference on Machine Learning",
month="July",year="2008",
abstract={ Hierarchical decomposition promises to help scale
reinforcement learning algorithms naturally to
real-world problems by exploiting their underlying
structure. Model-based algorithms, which provided the
first finite-time convergence guarantees for
reinforcement learning, may also play an important
role in coping with the relative scarcity of data in
large environments. In this paper, we introduce an
algorithm that fully integrates modern hierarchical
and model-learning methods in the standard
reinforcement learning setting. Our algorithm,
\textsc{R-maxq}, inherits the efficient model-based
exploration of the \textsc{R-max} algorithm and the
opportunities for abstraction provided by the MAXQ
framework. We analyze the sample complexity of our
algorithm, and our experiments in a standard
simulation environment illustrate the advantages of
combining hierarchies and models. },
wwwnote={<a href="http://icml2008.cs.helsinki.fi/">ICML 2008</a>},
}
Generated by bib2html.pl (written by Patrick Riley ) on Tue Nov 19, 2024 10:24:45