CHILD: A First Step Towards Continual Learning, Machine Learning Journal, vol. 28, 1997. Also appears as Chapter 11 in Learning to Learn, S. Thrun and L. Pratt, editors.

Continual learning is the constant development of complex behaviors with no final end in mind.  It is the process of learning ever more complicated skills by building on those skills already developed.  In order for learning at one stage of development to serve as the foundation for later learning, a continual-learning agent should learn hierarchically.  CHILD, an agent capable of Continual, Hierarchical, Incremental Learning and Development, accumulates useful behaviors in reinforcement environments by using the Temporal Transition Hierarchies learning algorithm.  This algorithm dynamically constructs a hierarchical, higher-order neural network that can learn to predict context-dependent temporal sequences.  CHILD can quickly solve complicated non-Markovian reinforcement-learning tasks and can then transfer its skills to similar but even more complicated tasks, learning these faster still.  This continual-learning approach is possible because Temporal Transition Hierarchies allow existing skills to be amended and augmented in precisely the same way that they were constructed in the first place.