next up previous
Next: Acknowledgements Up: LEARNING TO CONTROL FAST-WEIGHT Previous: Learning Temporary Variable Binding

Concluding Remarks

The system described above is a special case of a more general class of adaptive systems (which also includes conventional recurrent nets) which employ some parameterized memory function for changing a vector-valued memory structure and which employ some parameterized retrieval function for processing the contents of the memory structure and the current input. The only requirement is that the memory and retrieval functions be differentiable with respect to their internal parameters.

Such systems work because of the existence of the chain rule. Results as above (as well as other novel applications of the chain rule [Schmidhuber, 1991a][Schmidhuber, 1990a]) indicate that there may be additional interesting (yet undiscovered) ways of applying the chain rule for temporal credit assignment in adaptive systems.

Juergen Schmidhuber 2003-02-13

Back to Recurrent Neural Networks page