LSTM metalearner (Hochreiter, 2001)
LSTM, 5000 weights, 5 months training: metalearns fast online learning algorithm for quadratic functions f(x,y)=a1x2+a2y2+a3xy+a4x+a5y+a6 Huge time lags.
After metalearning, freeze weights.
Now use net: Select new f, feed training exemplars ...data/target/data/target/data... into input units, one at a time. After 30 exemplars the net predicts target inputs before it sees them. No weight changes! How?
Back to J. Schmidhuber's Recurrent neural network page