LSTM CAN SOLVE HARD LONG TIME LAG PROBLEMS

Sepp Hochreiter, TUM
Jürgen Schmidhuber, IDSIA

In M. C. Mozer, M. I. Jordan, T. Petsche, eds., Advances in Neural Information Processing Systems 9, NIPS'9, pages 473-479, MIT Press, Cambridge MA, 1997.

Abstract:

Standard recurrent nets cannot deal with long minimal time lags between relevant signals. Several recent NIPS papers propose alternative methods. We first show: problems used to promote various previous algorithms can be solved more quickly by random weight guessing than by the proposed algorithms. We then use LSTM, our own recent algorithm, to solve hard problems that can neither be quickly solved by random search nor by any other recurrent net algorithm we are aware of.

Back to Recurrent Neural Networks page