• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
Model-Selection for Non-Parametric Function Approximation in Continuous Control Problems: A Case Study in a Smart Energy
System.
Daniel Urieli and Peter
Stone.
In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases (ECML'13),
Sep 2013.
Official publisher version
[PDF]2.3MB [postscript]1.4MB [slides.pdf]3.6MB
This paper investigates the application of value-function-based reinforcement learning to a smart energy control system, specifically the task of controlling an HVAC system to minimize energy while satisfying residents' comfort requirements. In theory, value-function-based reinforcement learning methods can solve control problems such as this one optimally. However, since choosing an appropriate parametric representation of the value function turns out to be difficult, we develop an alternative method, which results in a practical algorithm for value function approximation in continuous state-spaces. To avoid the need to carefully design a parametric representation for the value function, we use a smooth non-parametric function approximator, specifically Locally Weighted Linear Regression (LWR). LWR is used within Fitted Value Iteration (FVI), which has met with several practical successes. However, for efficiency reasons, LWR is used with a limited sample-size, which leads to poor performance without careful tuning of LWR's parameters. We therefore develop an efficient meta-learning procedure that performs online model-selection and tunes LWR's parameters based on the Bellman error. Our algorithm is fully implemented and tested in a realistic simulation of the HVAC control domain, and results in significant energy savings.
@InProceedings{ECML13-urieli,
author = {Daniel Urieli and Peter Stone},
title = {Model-Selection for Non-Parametric Function Approximation in Continuous Control Problems: A Case Study in a Smart Energy System},
booktitle = {Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases (ECML'13)},
location = {Prague, Czech Republic},
month = {Sep},
year = {2013},
abstract = "
This paper investigates the application of value-function-based reinforcement
learning to a smart energy control system, specifically the task of
controlling an HVAC system to minimize energy while satisfying residents'
comfort requirements. In theory, value-function-based reinforcement learning
methods can solve control problems such as this one optimally. However,
since choosing an appropriate parametric representation of the value function
turns out to be difficult, we develop an alternative method, which results in
a practical algorithm for value function approximation in continuous
state-spaces. To avoid the need to carefully design a parametric
representation for the value function, we use a smooth non-parametric
function approximator, specifically Locally Weighted Linear Regression (LWR).
LWR is used within Fitted Value Iteration (FVI), which has met with several
practical successes. However, for efficiency reasons, LWR is used with a
limited sample-size, which leads to poor performance without careful tuning
of LWR's parameters. We therefore develop an efficient meta-learning
procedure that performs online model-selection and tunes LWR's parameters
based on the Bellman error. Our algorithm is fully implemented and tested in
a realistic simulation of the HVAC control domain, and results in significant
energy savings.
",
wwwnote={Official <a href="http://link.springer.com/chapter/10.1007/978-3-642-40988-2_5">publisher version</a>},
}
Generated by bib2html.pl (written by Patrick Riley ) on Sun Nov 24, 2024 20:24:55