Incremental Development of Complex Behaviors through Automatic Construction of Sensory-motor Hierarchies, from the proceedings of the Eighth International Workshop (ML91), 1991.

This paper addresses the issue of continual, incremental development of behaviors in reactive agents. The reactive agents are neural-network based and use reinforcement learning techniques.

A continually developing system is one that is constantly capable of extending its repertoire of behaviors. An agent increases its repertoire of behaviors in order to increase its performance in and understanding of its environment.  Continual development requires an unlimited growth potential; that is, it requires a system that can constantly augment current behaviors with new behaviors, perhaps using the current ones as a foundation for those that come next.  It also requires a process for organizing behaviors in meaningful ways and a method for assigning credit properly to sequences of behaviors, where each behavior may itself be an arbitrarily long sequence.

The solution proposed here is hierarchical and bottom up.  I introduce a new kind of neuron (termed a ''bion''), whose characteristics permit it to be automatically constructed into sensory-motor hierarchies as determined by experience.  The bion is being developed to resolve the problems of incremental growth, temporal history limitation, network organization, and credit assignment among component behaviors.