Table of Contents
Long ShortTerm Memory: Tutorial on LSTM Recurrent Networks
Tutorial covers the following LSTM journal publications:
Even static problems may profit from recurrent neural networks (RNNs), e.g., parity problem: number of 1 bits odd? 9 bit feedforward NN:
Parity problem, sequential: 1 bit at a time
Other sequential problems
Other sequence learners?
Gradientbased RNNs: ? wish / ? program
1980s: BPTT, RTRL  gradients based on “unfolding” etc. (Williams, Werbos, Robinson)
1990s: Time Lags!
Exponential Error Decay
Training: forget minimal time lags > 10!
Constant Error Flow!
Basic LSTM unit: linear integrator
Long ShortTerm Memory (LSTM)
One possible LSTM cell (original)
LSTM cell (current standard)
PPT Slide
Mix LSTM cells and others
Mix LSTM cells and others
Also possible: LSTM memory blocks: error carousels may share gates
Example: no forget gates; 2 connected blocks, 2 cells each
Example with forget gates
Next: LSTM Pseudocode
PPT Slide
Experiments: first some LSTM limitations
“True” Sequence Experiments LSTM in a league by itself
Regular Grammars: LSTM vs Simple RNNs (Elman 1988) & RTRL / BPTT (Zipser & Smith)
Contextfree / Contextsensitive Languages
What this means:
Typical evolution of activations
Storing & adding real values
Noisy temporal order
Noisy temporal order II
Learning to compose music with RNNs?
Step 1: can LSTM learn precise timing?
Selfsustaining Oscillation
Step 2: Learning the Blues (Eck, 2002)
Learning to Learn?
Learning to learn
LSTM metalearner (Hochreiter, 2001)
LSTM metalearner: How?
LSTM metalearner
Learning to Learn?
Some day
Reinforcement Learning with RNNs
Reinforcement Learning RNNs II
Using LSTM for POMDPs (Bakker, 2001)
LSTM to approximate value function of reinforcement learning (RL) algorithm
Test problem 1: Longterm dependency Tmaze with noisy observations
Test problem 2: partially observable, multimode pole balancing
Results
Ongoing: Reinforcement Learning Robots Using LSTM
PPT Slide
PPT Slide

Author: J. Schmidhuber
Email: juergen@idsia.ch
Home Page: http://www.idsia.ch/~juergen
