by Peter Stone
MIT Press, 2000.
ISBN: 0262194384
Available from MIT Press and amazon.com
This book is based upon my
Ph.D. thesis
(Computer Science Department, Carnegie Mellon University, 1998)
Description
First, it describes an architecture within which a flexible team structure allows member agents to decompose a task into flexible roles and to switch roles while acting. Second, it presents layered learning, a general-purpose machine-learning method for complex domains in which learning a mapping directly from agents' sensors to their actuators is intractable with existing machine-learning methods. Third, the book introduces a new multiagent reinforcement learning algorithm--team-partitioned, opaque-transition reinforcement learning (TPOT-RL)--designed for domains in which agents cannot necessarily observe the state-changes caused by other agents' actions. The final contribution is a fully functioning multiagent system that incorporates learning in a real-time, noisy domain with teammates and adversaries--a computer-simulated robotic soccer team.
Peter Stone's work is the basis for the CMUnited Robotic Soccer Team, which has dominated recent RoboCup competitions. RoboCup not only helps roboticists to prove their theories in a realistic situation, but has drawn considerable public and professional attention to the field of intelligent robotics. The CMUnited team won the 1999 Stockholm simulator competition, outscoring its opponents by the rather impressive cumulative score of 110-0.
Contents
Chapter 1: Introduction
1.1 Motivation
1.2 Objectives and Approach
1.3 Contributions
1.4 Reader's Guide to the Book
Chapter 2: Substrate Systems
2.1 Overview
2.2 The RoboCup Soccer Server
2.3 The CMUnited-97 Real Robots
2.4 Network Routing
Chapter 3: Team Member Agent Architecture
3.1 Periodic Team Synchronization (PTS) Domains
3.2 Architecture Overview
3.3 Teamwork Structure
3.4 Communication Paradigm
3.5 Implementation in Robotic Soccer
3.6 Results
3.7 Transfer to Real Robots
3.8 Discussion and Related Work
Chapter 4: Layered Learning
4.1 Principles
4.2 Formalism
4.3 Instantiation in Simulated Robotic Soccer
4.4 Discussion
4.5 Related Work
Chapter 5: Learning an Individual Skill
5.1 Ball Interception in the Soccer Server
5.2 Training
5.3 Results
5.4 Discussion
5.5 Related Work
Chapter 6: Learning a Multi-Agent Behavior
6.1 Decision Tree Learning for Pass Evaluation
6.2 Using the Learned Behaviors
6.3 Scaling up to Full Games
6.4 Discussion
6.5 Related Work
Chapter 7: Learning a Team Behavior
7.1 Motivation
7.2 TPOT-RL
7.3 TPOT-RL Applied to Simulated Robotic Soccer
7.4 TPOT-RL Applied to Network Routing
7.5 Discussion
7.6 Related Work
Chapter 8: Competition Results
8.1 Pre-RoboCup-96
8.2 MiroSot-96
8.3 RoboCup-97
8.4 RoboCup-98
8.4 RoboCup-99
8.6 Lessons Learned from Competitions
Chapter 9: Related Work
9.1 MAS from an ML Perspective
9.2 Robotic Soccer
Chapter 10: Conclusion
10.1 Contributions
10.2 Future Directions
10.3 Concluding Remarks
Appendices
A List of Acronyms
B Robotic Soccer Agent Skills
B.1 CMUnited-98 Simulator Agent Skills
B.2 CMUnited-97 Small-Robot Skills
C CMUnited-98 Simulator Team Behavior Modes
C.1 Conditions
C.2 Effects
D CMUnited Simulator Team Source Code
On-line Appendix
Information, source, and executables pertaining to the CMUnited-99 simulator team are also available from the CMUnited-99 team page.