next up previous
Next: Acknowledgements Up: Reinforcement Learning in Markovian Previous: EXPERIMENTS

SOME LIMITATIONS OF THE APPROACHES

1. The recurrent network algorithms are not local in space.

2. As with all gradient descent algorithms there is the problem of local minima. This paper does not offer any solutions to this problem.

3. More severe limitations of the algorithm are inherent problems of the concepts of `gradient descent through time' and adaptive critics. Neither gradient descent nor adaptive critics are practical when there are long time lags between actions and ultimate consequences. For this reason, first steps are made in [9] towards adaptive sub-goal generators and adaptive `causality detectors'.



Subsections

Juergen Schmidhuber 2003-02-25


Back to Reinforcement Learning POMDP page