Long Short-Term Memory: Tutorial on LSTM Recurrent Networks

1/14/2003


Click here to start


Table of Contents

Long Short-Term Memory: Tutorial on LSTM Recurrent Networks

Tutorial covers the following LSTM journal publications:

Even static problems may profit from recurrent neural networks (RNNs), e.g., parity problem: number of 1 bits odd? 9 bit feedforward NN:

Parity problem, sequential: 1 bit at a time

Other sequential problems

Other sequence learners?

Gradient-based RNNs: ? wish / ? program

1980s: BPTT, RTRL - gradients based on “unfolding” etc. (Williams, Werbos, Robinson)

1990s: Time Lags!

Exponential Error Decay

Training: forget minimal time lags > 10!

Constant Error Flow!

Basic LSTM unit: linear integrator

Long Short-Term Memory (LSTM)

One possible LSTM cell (original)

LSTM cell (current standard)

PPT Slide

Mix LSTM cells and others

Mix LSTM cells and others

Also possible: LSTM memory blocks: error carousels may share gates

Example: no forget gates; 2 connected blocks, 2 cells each

Example with forget gates

Next: LSTM Pseudocode

PPT Slide

Experiments: first some LSTM limitations

“True” Sequence Experiments LSTM in a league by itself

Regular Grammars: LSTM vs Simple RNNs (Elman 1988) & RTRL / BPTT (Zipser & Smith)

Contextfree / Contextsensitive Languages

What this means:

Typical evolution of activations

Storing & adding real values

Noisy temporal order

Noisy temporal order II

Learning to compose music with RNNs?

Step 1: can LSTM learn precise timing?

Self-sustaining Oscillation

Step 2: Learning the Blues (Eck, 2002)

Learning to Learn?

Learning to learn

LSTM metalearner (Hochreiter, 2001)

LSTM metalearner: How?

LSTM metalearner

Learning to Learn?

Some day

Reinforcement Learning with RNNs

Reinforcement Learning RNNs II

Using LSTM for POMDPs (Bakker, 2001)

LSTM to approximate value function of reinforcement learning (RL) algorithm

Test problem 1: Long-term dependency T-maze with noisy observations

Test problem 2: partially observable, multi-mode pole balancing

Results

Ongoing: Reinforcement Learning Robots Using LSTM

PPT Slide

PPT Slide

Author: J. Schmidhuber

Email: juergen@idsia.ch

Home Page: http://www.idsia.ch/~juergen