• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
Learning Predictive State Representations.
Satinder Singh, Michael
L. Littman, Nicholas K. Jong, David
Pardoe, and Peter Stone.
In Proceedings of the Twentieth International
Conference on Machine Learning, August 2003.
ICML-2003
[PDF]316.3kB [postscript]319.2kB
We introduce the first algorithm for learning predictive state representations PSRs, which are a way of representing the state of a controlled dynamical system. The state representation in a PSR is a vector of predictions of tests, where tests are sequences of actions and observations said to be true if and only if all the observations occur given that all the actions are taken. The problem of finding a good PSR---one that is a sufficient statistic for the dynamical system---can be divided into two parts: 1) discovery of a good set of tests, and 2) learning to make accurate predictions for those tests. In this paper, we present detailed empirical results using a gradient-based algorithm for addressing the second problem. Our results demonstrate several sample systems in which the algorithm learns to make correct predictions and several situations in which the algorithm is less successful. Our analysis reveals challenges that will need to be addressed in future PSR learning algorithms.
@InProceedings{ICML03,
author="Satinder Singh and Michael L.~Littman and Nicholas K.~Jong and David Pardoe and Peter Stone",
title="Learning Predictive State Representations",
booktitle="Proceedings of the Twentieth International Conference on Machine Learning",
year="2003",month="August",
abstract={
We introduce the first algorithm for learning
predictive state representations PSRs, which are a
way of representing the state of a controlled
dynamical system. The state representation in a PSR
is a vector of predictions of tests, where tests are
sequences of actions and observations said to be
true if and only if all the observations occur given
that all the actions are taken. The problem of
finding a good PSR---one that is a sufficient
statistic for the dynamical system---can be divided
into two parts: 1) discovery of a good set of tests,
and 2) learning to make accurate predictions for
those tests. In this paper, we present detailed
empirical results using a gradient-based algorithm
for addressing the second problem. Our results
demonstrate several sample systems in which the
algorithm learns to make correct predictions and
several situations in which the algorithm is less
successful. Our analysis reveals challenges that
will need to be addressed in future PSR learning
algorithms.
},
wwwnote={<a href="http://www.hpl.hp.com/conferences/icml03/">ICML-2003</a>},
}
Generated by bib2html.pl (written by Patrick Riley ) on Sun Nov 24, 2024 20:24:56