• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
Data-Efficient Policy Evaluation Through Behavior Policy Search.
Josiah
P. Hanna, Yash Chandak, Philip S. Thomas, Martha White,
Peter Stone, and Scott Niekum.
Journal
of Machine Learning Research, 2024.
Official version on publisher's
website
We consider the task of evaluating a policy for a Markov decision process (MDP).The standard unbiased technique for evaluating a policy is to deploy the policyand observe its performance. We show that the data collected from deploying adifferent policy, commonly called the behavior policy, can be used to produceunbiased estimates with lower mean squared error than this standard technique. Wederive an analytic expression for the optimal behavior policy --- the behaviorpolicy that minimizes the mean squared error of the resulting estimates. Becausethis expression depends on terms that are unknown in practice, we propose a novelpolicy evaluation sub-problem, behavior policy search: searching for a behaviorpolicy that reduces mean squared error. We present a behavior policy searchalgorithm and empirically demonstrate its effectiveness in lowering the meansquared error of policy performance estimates.
@Article{Hanna_JMLR2024,
author = {Josiah P. Hanna and Yash Chandak and Philip S. Thomas and Martha White and Peter Stone and Scott Niekum},
title = {Data-Efficient Policy Evaluation Through Behavior Policy Search},
journal = {Journal of Machine Learning Research},
year = {2024},
abstract = {We consider the task of evaluating a policy for a Markov decision process (MDP).
The standard unbiased technique for evaluating a policy is to deploy the policy
and observe its performance. We show that the data collected from deploying a
different policy, commonly called the behavior policy, can be used to produce
unbiased estimates with lower mean squared error than this standard technique. We
derive an analytic expression for the optimal behavior policy --- the behavior
policy that minimizes the mean squared error of the resulting estimates. Because
this expression depends on terms that are unknown in practice, we propose a novel
policy evaluation sub-problem, behavior policy search: searching for a behavior
policy that reduces mean squared error. We present a behavior policy search
algorithm and empirically demonstrate its effectiveness in lowering the mean
squared error of policy performance estimates.
},
wwwnote={<a href="http://jmlr.org/papers/v25/21-0346.html">Official version</a> on publisher's website},
}
Generated by bib2html.pl (written by Patrick Riley ) on Wed Jan 15, 2025 08:40:49