Research

Learning Agents Research Group

I am director of LARG, the Learning Agents Research Group in the UTCS AI Lab. All of the current research projects and many of the past projects listed on this page were done in this group. All of my current and past students are (or were) a part of this group.
A couple of videos with overviews of LARG research were produced by Benefunder and by the Natural Sciences Council.
LARG research is supported in part by grants from the National Science Foundation (CNS-1330072, CNS-1305287), ONR (21C184-01), AFRL (FA8750-14-1-0070), AFOSR (FA9550-14-1-0087), and TXDOT (0-6838, 0-6847).
[ LARG Publications ]

Reinforcement Learning

A large part of the lab's research focus is on developing new reinforcement learning algorithms with particular focus on cyber-physical systems and scaling up to large-scale applications.
Representative Publication:
TEXPLORE: Real-Time Sample-Efficient Reinforcement Learning for Robots (Machine Learning, 2013).
[ Full List of Relevant Publications ]

Multiagent Systems

One main theme of the lab is the study of interactions among independent autonomous agents (including robots), be they teammates, adversaries, or neither. Some of our research on this topic contributes to and makes use of game theory.
Representative Publication:
Multiagent Systems: A survey from a machine learning perspective (Autonomous Robots, 2000).
[ Full List of Relevant Publications ]

Robot Soccer

One of the main application domains used throughout the lab is robot soccer, both in simulation and on real robots. We have won multiple RoboCup championships.
Representative Publication:
Reinforcement Learning for RoboCup-Soccer Keepaway (Adaptive Behavior, 2005).
[ Full List of Relevant Publications ]
[ Project Page ]
[ Watch Video ]

Building-Wide Intelligence

The mission the Building-Wide Intelligence Project is to develop fully autonomous mobile robots that can become a permanent part of a building's environment.
Representative Publication:
Leading the Way: An Efficient Multi-robot Guidance System (AAMAS 2015).
[ Full List of Relevant Publications ]
[ Project Page ]

Ad Hoc Teamwork

We study algorithms that enable an individual agent to collaborate with previously unknown team members.
Representative Publication:
Ad Hoc Autonomous Agent Teams: Collaboration without Pre-Coordination (AAAI, 2010).
[ Full List of Relevant Publications ]
[ Project Page ]

Autonomous Traffic Management

We introduced a novel,  efficient multiagent mechanism for future autonomous vehicles to navigate intersections.
Representative Publication:
A Multiagent Approach to Autonomous Intersection Management (JAIR, 2008).
[ Full List of Relevant Publications ]
[ Project Page ]
[ Watch Video ]

Multiagent Learning (MAL)

We study learning algorithms that perform well against other learning agents.
Representative Publication:
Convergence, Targeted Optimality and Safety in Multiagent Learning (ICML, 2010).

Transfer Learning

We have developed algorithms from transfering knowledge from a previously learned task to a similar, but different, new learning task. We focus particularly on reinforcement learning tasks.
Representative Publication:
Transfer Learning via Inter-Task Mappings for Temporal Difference Learning (JMLR, 2007).
[ Full List of Relevant Publications ]

Social Agents

We finished in 2nd place in the 2007 RoboCup@Home competition.
Representative Publication:
Inter-Classifier Feedback for Human-Robot Interaction in a Domestic Setting (JoPhA, 2008).
[ Full List of Relevant Publications ]
[ Project Page ]
[ Watch Video ]

Machine Learning for Robot Navigation

We are investigating methods for using machine learning to improve classical robot navigation algorithms.
Representative Publication:
APPLD: Adaptive Planner Parameter Learning from Demonstration (IROS, 2020).
[ Adaptive Planner Parameter Learning project page ]
[ Learning from Hallucination project page ]

Trading Agents

Another main application domain has been autonomous trading agents, including supply chain management, ad auctions, and mechanism design. We have won multiple Trading Agent Competitions.
Representative Publication:
TacTex-2005: A Champion Supply Chain Management Agent (AAAI, 2006).
[ Full List of Relevant Publications ]
[ Project Page ]

Teaching an Agent Manually via Evaluative Reinforcement (TAMER)

The TAMER project seeks to create agents which can be effectively taught behaviors by lay people using positive and negative feedback signals (akin to "shaping" by reward and punishment in animal training).
Representative Publication:
Framing reinforcement learning from human reward: Reward positivity, temporal discounting, episodicity, and performance (AIJ, 2015).
[ Full List of Relevant Publications ]
[ Project Page ]
[ Watch Video ]

Autonomous Driving

We have a full-size autonomous vehicle that we use to study autonomous driving in the real world.
Representative Publication:
Multiagent Interactions in Urban Driving (JoPhA, 2008).
[ Full List of Relevant Publications ]
[ Project Page ]
[ Watch Video ]

Robot Vision

We developed algorithms suitable for real-time visual sensing of the physical world on mobile robots.
Representative Publication:
Color Learning on a Mobile Robot: Towards Full Autonomy under Changing Illumination (IJCAI, 2007).
[ Full List of Relevant Publications ]
[ Project Page ]

Learned Quadruped Robot Walking

We enabled an Aibo robot to learn to walk faster than was previously possible.
Representative Publication:
Machine Learning for Fast Quadrupedal Locomotion (AAAI, 2004).
[ Full List of Relevant Publications ]
[ Project Page ]
[ Watch Video ]

General Game Playing

We participated successfully in the first few general game playing competitions.
Representative Publication:
Automatic Heuristic Construction in a Complete General Game Player (AAAI, 2006).
[ Full List of Relevant Publications ]

Autonomic Computing

We developed machine learning approaches for computer systems applications.
Representative Publication:
Machine Learning for On-Line Hardware Reconfiguration (IJCAI, 2007).
[ Full List of Relevant Publications ]

Developmental Robotics

We have developed methods for robots to autonomously discover models of their own sensor and actuators.
Representative Publication:
Towards Autonomous Sensor and Actuator Model Induction on a Mobile Robot (Connection Science, 2006).
[ Full List of Relevant Publications ]
[ Project Page ]
[ Watch Video ]

Predictive State Representations

We have contributed to the literature on representating an agent's state entirely via predictions of its future sensations as a function of its possible actions. Thus it does not need to reason explicitly about objects.
Representative Publication:
Learning Predictive State Representations (ICML, 2003).
[ Full List of Relevant Publications ]

Layered Learning

My Ph.D. thesis introduced a general hierarchical machine learning paradigm by which complex tasks can be learned via several interacting learned layers.
Representative Publication:
Concurrent Layered Learning (AAMAS, 2003).
[ Full List of Relevant Publications ]

Planning

My first research as a Ph.D. student was within the area of classical AI planning. Some of our current research falls in the area of modern planning and scheduling
Representative Publication:
FLECS: Planning with a Flexible Commitment Strategy (JAIR, 1995).
[ Full List of Relevant Publications ]