Faraz Torabi
faraztrb [at] cs.utexas.edu

I recently defended my PhD from Learning Agents Research Group in the Computer Science Department at the University of Texas at Austin, where I worked with Prof. Peter Stone. My PhD research focused on imitation learning where, in general, the goal is for an autonomous agent to learn a task by observing another, more expert agent performs that task. I was also a part of the UT Austin Villa RoboCup 3D simulation team where I helped improving the passing strategy and worked on improving the kick and walk skills of the agents.

I am currently on the job market for research positions in industry. In general, I am interested in machine learning, reinforcement learning, and imitation learning.

CV  /  Google Scholar  /  Dissertation

we propose an imitation from observation (IfO) algorithm, Data-Efficient Adversarial Learning for Imitation from Observation (DEALIO), that incorporates ideas from model-based reinforcement learning with adversarial methods for IfO in order to increase the data efficiency of these methods without sacrificing performance. Specifically, we consider time-varying linear Gaussian policies, and propose a method that integrates the linear-quadratic regulator with path integral policy improvement into an existing adversarial IfO framework. The result is a more data-efficient IfO algorithm with better performance, which we show empirically in four simulation domains: using far fewer interactions with the environment, the proposed method exhibits similar or better performance than the existing technique.

We propose a new imitation learning algorithm called SILEM (Skeletal feature compensation for Imitation Learning with Embodiment Mismatch) that can help reliably train a learner to imitate an expert with a different body. Central to SILEM is a learned affine transform that compensates for differences in the skeletal features (e.g. joint angles, height of head, etc.) derived from the expert and learner. We also provide qualitative and quantitative results, teaching simulated humanoid agents, including Atlas from Boston Dynamics, to walk by observing human demonstrations.

We propose a new technique for performing integrated IL and RL called reinforced inverse dynamics modeling (RIDM). RIDM uses recent ideas from model-based imitation from observation (IfO) to find a controller that is able to both mimic the demonstrated single state trajectory and also maximize an external environment reward signal. We experimentally compare our technique to a reasonable baseline algorithm in relevant scenarios on six MuJoCo domains, finding that it produces state-of-the-art results. We also apply our algorithm to simulated robot soccer skill learning and a UR5 physical robot arm.

We build upon our previous work, GAIfO, proposing an algorithm that uses a GAN-like architecture to learn tasks perform IfO directly from videos. Unlike our prior work, however, our method also uses proprioceptive information from the imitating agent during the learning process. We show that the addition of such information will improve both learning speed and the final performance of the imitator in several standard simulation domains.

Conventionally, in imitation learning the imitator has access to both state and action information generated by an expert performing the task. However, requiring the action information prevents imitation learning from a large number of existing valuable learning resources such as online videos of humans performing tasks. To overcome this issue, the specific problem of imitation from observation (IfO) has recently garnered a great deal of attention, in which the imitator only has access to the state information (e.g., video frames) generated by the expert. In this paper, we provide a literature review of methods developed for IfO, and then point out some open research problems and potential future work.

Reinforcement learning agents can learn to solve sequential decision tasks by interacting with the environment. Human knowledge of how to solve these tasks can be incorporated using imitation learning, where the agent learns to imitate human demonstrated decisions. However, human guidance is not limited to the demonstrations. Other types of guidance could be more suitable for certain tasks and require less human effort. In this paper we provide a high-level overview of five recent learning frameworks that primarily rely on human guidance other than conventional, step-by-step action demonstrations. We review the motivation, assumption, and implementation of each framework.

We propose a two-phase, autonomous imitation from observation technique called behavioral cloning from observation (BCO). We allow the agent to acquire experience in a self-supervised fashion. This experience is used to develop a model which is then utilized to learn a particular task by observing an expert perform that task without the knowledge of the specific actions taken.

Website template credits.