Next: Acknowledgements
Up: LEARNING TO CONTROL FAST-WEIGHT
Previous: Learning Temporary Variable Binding
The system described above is a special case of a more general
class of adaptive systems (which also includes conventional
recurrent nets) which employ some parameterized memory function
for changing a vector-valued memory structure and
which employ some parameterized retrieval function
for processing the contents of the memory structure
and the current input.
The only requirement is that the memory and retrieval functions be
differentiable with respect to their internal parameters.
Such systems work because of the existence of
the chain rule. Results as above (as well as other novel
applications of the chain rule
[Schmidhuber, 1991a][Schmidhuber, 1990a])
indicate that there may be additional interesting (yet undiscovered)
ways of applying the
chain rule for temporal credit assignment in adaptive systems.
Juergen Schmidhuber
2003-02-13
Back to Recurrent Neural Networks page