Learning to learn
Schmidhuber (1993): a self-referential weight matrix. RNN can read and actively change its own weights; runs weight change algorithm on itself; uses gradient-based metalearning algorithm to compute better weight change algorithm.
Did not work well in practice, because standard RNNs were used instead of LSTM.
But Hochreiter recently used LSTM for metalearning (2001) and obtained astonishing results.
Back to J. Schmidhuber's Recurrent neural network page