Simulation Leagues
In the 2D soccer simulation league, teams of independently-controlled software agents operate in the RoboCup Soccer Server System, a rich and challenging domain that incorporates many realistic features. The 2D simulation league incorporates features such as noisy sensors and actuators, limited vision, models of stamina, and limited inter-agent communication. Research challenges include strategic teamwork and multiagent learning. The 3D simulation league involves programming a humanoid robot with 20 degrees of freedom at a low level of control, and offers a multitude of challenges, including the development robotic skills such as bipedal locomotion and kicking. Our efforts in the 3D simulation league are documented in the UT Austin Villa 3D Simulation page.
We participate both in the main competition and the coach competition, in which an omniscient coach agent aims to improve the performance of a team of agents created by other programmers by giving strategic suggestions via a standardized coach language.
In addition to the competition, we have used the simulator for other research projects including Keepaway. In this domain, a team of players tries to maintain possession of the ball in a small rectangular playing region while the opposing team tries to gain possession. We have successfully applied reinforcement learning techniques to this problem. We have released a player framework to allow others to try out their learning algorithms in this domain. For information, visit our Keepaway site. In related work, we extend learning in Keepaway to include the behavior of teammates in moving to positions on the field; we describe this work on our Keepaway GetOpen page.
As a step towards the ongoing challenge of learning complete team behavior for RoboCup simulated soccer, we have recently extended Keepaway to a more challenging and realistic task, Half Field Offense. Half Field Offense models a typical attack scenario in soccer, in which the offense of one team must get past the defense of the opposing team and attempt to shoot goals. We have improved upon the reinforcement learning algorithm used for Keepaway to scale to the harder Half Field Offense task. A complete description is provided in our Half Field Offense site.
