of Complex Behaviors through Automatic Construction of Sensory-motor
Hierarchies, from the proceedings of the Eighth International
Workshop (ML91), 1991.
This paper addresses the issue of continual, incremental development of
behaviors in reactive agents. The reactive agents are neural-network
based and use reinforcement learning techniques.
A continually developing system is one that is constantly capable of
extending its repertoire of behaviors. An agent increases its
repertoire of behaviors in order to increase its performance in and
understanding of its environment. Continual development requires
an unlimited growth potential; that is, it requires a system that can
constantly augment current behaviors with new behaviors, perhaps using
the current ones as a foundation for those that come next. It
also requires a process for organizing behaviors in meaningful ways and
a method for assigning credit properly to sequences of behaviors, where
each behavior may itself be an arbitrarily long sequence.
The solution proposed here is hierarchical and bottom up. I
introduce a new kind of neuron (termed a ''bion''), whose
characteristics permit it to be automatically constructed into
sensory-motor hierarchies as determined by experience. The bion
is being developed to resolve the problems of incremental growth,
temporal history limitation, network organization, and credit
assignment among component behaviors.