Selected conference publications on LSTM and other RNNs / feedback networks:
52.
M. Stollenga, W. Beyon, M. Liwicki, J. Schmidhuber. Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation. Advances in Neural Information Processing Systems (NIPS), 2015, in press.
Preprint: arxiv:1506.07452.
51.
K. Greff, R. K. Srivastava, J. Schmidhuber. Training Very Deep Networks. Advances in Neural Information Processing Systems (NIPS), 2015, in press.
Preprint: arxiv:1505.00387.
50. J. Koutnik, K. Greff, F. Gomez, J. Schmidhuber. A Clockwork RNN. Proc. 31st International Conference on Machine Learning (ICML), p. 1845-1853, Beijing, 2014. Preprint arXiv:1402.3511 [cs.NE].
49.
M. Stollenga, J.Masci, F. Gomez, J. Schmidhuber.
Deep Networks with Internal Selective Attention through Feedback Connections.
Preprint arXiv:1407.3068 [cs.CV].
Advances in Neural Information Processing Systems (NIPS), 2014.
48.
J. Koutnik, G. Cuccu, J. Schmidhuber, F. Gomez.
Evolving Large-Scale Neural Networks for Vision-Based Reinforcement Learning.
In Proceedings of the Genetic and Evolutionary
Computation Conference (GECCO), Amsterdam, 2013.
PDF.
47.
J. Koutnik, G. Cuccu, J. Schmidhuber, F. Gomez.
Evolving Large-Scale Neural Networks for Vision-Based TORCS.
In Foundations of Digital Games (FDG), Chania, Crete, 2013.
PDF.
46.
R. K. Srivastava, B. Steunebrink, M. Stollenga, J. Schmidhuber
Continually Adding Self-Invented Problems to the Repertoire: First Experiments with PowerPlay.
Proc. IEEE Conference on Development and Learning / EpiRob 2012
(ICDL-EpiRob'12), San Diego, 2012. PDF.
45.
F. Gomez, J. Koutnik, J. Schmidhuber.
Compressed Network Complexity Search.
In C. Coello Coello, V. Cutello, K. Deb, S. Forrest, G. Nicosia, M. Pavone, eds.,
12th Int. Conf. on Parallel Problem Solving from Nature - PPSN XII,
Taormina, 2012. Nominated for best paper award.
PDF.
44.
R. K. Srivastava, F. Gomez, J. Schmidhuber.
Generalized Compressed Network Search.
In C. Coello Coello, V. Cutello, K. Deb, S. Forrest, G. Nicosia, M. Pavone, eds.,
12th Int. Conf. on Parallel Problem Solving from Nature - PPSN XII,
Taormina, 2012.
PDF.
43.
M. Ring, T. Schaul, J. Schmidhuber.
The Two-Dimensional Organization of Behavior.
In Proc. Joint IEEE International Conference on Development and Learning (ICDL) and on Epigenetic Robotics (ICDL-EpiRob 2011), Frankfurt, 2011.
PDF.
42.
J. Schmidhuber, D. Ciresan, U. Meier, J. Masci, A. Graves.
On Fast Deep Nets for AGI Vision.
In Proc. Fourth Conference on Artificial General Intelligence (AGI-11),
Google, Mountain View, California, 2011.
PDF.
41.
L. Gisslen, M. Luciw, V. Graziano, J. Schmidhuber.
Sequential Constant Size Compressors and Reinforcement Learning.
In Proc. Fourth Conference on Artificial General Intelligence (AGI-11),
Google, Mountain View, California, 2011.
PDF.
Kurzweil Prize for Best AGI Paper 2011.
40.
T. Glasmachers, T. Schaul, Sun Yi, D. Wierstra, J. Schmidhuber.
Exponential Natural Evolution Strategies.
Proceedings of the Genetic and Evolutionary Computation Conference
(GECCO-2010), Portland, 2010.
PDF.
GECCO 2010 best paper nomination.
39.
J. Koutnik, F. Gomez, J. Schmidhuber (2010). Evolving Neural Networks in Compressed Weight Space. Proceedings of the Genetic and Evolutionary Computation Conference
(GECCO-2010), Portland, 2010.
PDF.
38.
J. Koutnik, F. Gomez, J. Schmidhuber.
Searching for Minimal Neural Networks in Fourier Space.
The 3rd Conference on Artificial General Intelligence (AGI-10), 2010.
PDF.
37.
M. Grüttner, F. Sehnke, T. Schaul, J. Schmidhuber.
Multi-Dimensional Deep Memory Go-Player for Parameter Exploring Policy Gradients.
Proceedings of the International Conference on Artificial Neural Networks (ICANN-2010),
Greece, 2010.
36.
A. Graves, J. Schmidhuber.
Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks.
Advances in Neural Information Processing Systems 22, NIPS'22, p 545-552,
Vancouver, MIT Press, 2009.
PDF.
35.
J. Bayer, D. Wierstra, J. Togelius, J. Schmidhuber.
Evolving memory cell structures for sequence learning.
Proceedings of the 19th International Conference on Artificial Neural Networks (ICANN-09),
Cyprus, 2009.
PDF.
34.
J. Unkelbach, S. Yi, J. Schmidhuber.
An EM based training algorithm for recurrent neural networks.
Proceedings of the 19th International Conference on Artificial Neural Networks (ICANN-09),
Cyprus, 2009.
PDF.
33.
A. Graves, S. Fernandez,M. Liwicki, H. Bunke, J. Schmidhuber.
Unconstrained online handwriting recognition with recurrent neural networks.
Advances in Neural Information Processing Systems 21, NIPS'21,
p 577-584, 2008, MIT Press,
Cambridge, MA, 2008.
PDF.
32.
T. Rückstiess, M. Felder, J. Schmidhuber.
State-Dependent Exploration for Policy Gradient Methods.
19th European Conference on Machine Learning ECML,
2008.
PDF.
31.
T. Schaul and J. Schmidhuber. A Scalable Neural Network Architecture for Board Games.
Proceedings of the 2008 IEEE Symposium on Computational Intelligence in Games CIG-2008,
Perth, Australia, 2008, in press.
PDF.
30.
F. Sehnke, C. Osendorfer, T. Rückstiess, A. Graves, J. Peters, and J. Schmidhuber.
Policy gradients with parameter-based exploration for control.
In J. Koutnik V. Kurkova, R. Neruda, editors,
Proceedings of the
International Conference on Artificial Neural Networks ICANN-2008
ICANN 2008, Prague, LNCS 5163, pages 387-396. Springer-Verlag Berlin Heidelberg, 2008.
PDF.
29.
D. Wierstra, T. Schaul, J. Peters, J. Schmidhuber. Fitness Expectation Maximization.
Proceedings of Parallel Problem Solving from Nature PPSN-2008, Dortmund, 2008.
PDF.
28.
D. Wierstra, T. Schaul, J. Peters, J. Schmidhuber.
Natural Evolution Strategies.
Proceedings of IEEE Congress on Evolutionary Computation CEC-2008, Hongkong, 2008.
PDF.
27.
D. Wierstra, J. Schmidhuber.
Policy Gradient Critics.
18th European Conference on Machine Learning ECML,
Warszaw, 2007.
PDF.
26.
M. Liwicki, A. Graves, H. Bunke, J. Schmidhuber. A novel approach
to on-line handwriting recognition based on bidirectional
long short-term memory networks. 9th International Conference
on Document Analysis and Recognition, 2007.
PDF.
25.
S. Fernandez, A. Graves, J. Schmidhuber. An application of
recurrent neural networks to discriminative keyword
spotting.
Intl. Conf. on Artificial Neural Networks ICANN'07,
2007.
PDF.
24.
A. Graves, S. Fernandez, J. Schmidhuber. Multi-Dimensional Recurrent
Neural Networks.
Intl. Conf. on Artificial Neural Networks ICANN'07,
2007.
Preprint: arxiv:0705.2011.
PDF.
23.
D. Wierstra, A. Foerster, J. Schmidhuber. Solving Deep Memory POMDPs
with Recurrent Policy Gradients.
Intl. Conf. on Artificial Neural Networks ICANN'07,
2007.
22.
A. Foerster, A. Graves, J. Schmidhuber.
RNN-based Learning of Compact Maps for Efficient Robot Localization.
15th European Symposium on Artificial Neural Networks,
ESANN, Bruges, Belgium, 2007
PDF.
21.
S. Fernandez, A. Graves, J. Schmidhuber.
Sequence labelling in structured domains with
hierarchical recurrent neural networks. In Proc.
20th International Joint Conference on Artificial Intelligence
(IJCAI 07), p. 774-779, Hyderabad, India, 2007 (talk).
PDF.
20.
F. Gomez, J. Schmidhuber, and R. Miikkulainen (2006).
Efficient Non-Linear Control through Neuroevolution.
Proceedings of the European Conference
on Machine Learning (ECML-06, Berlin).
PDF.
19.
H. Mayer, F. Gomez, D. Wierstra, I. Nagy, A. Knoll, and J. Schmidhuber (2006).
A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent
Neural Networks. Proceedings of the International Conference on
Intelligent Robotics and Systems (IROS-06, Beijing).
PDF.
18.
A. Graves, S. Fernandez, F. Gomez, J. Schmidhuber.
Connectionist Temporal Classification: Labelling Unsegmented Sequence
Data with Recurrent Neural Networks. Proceedings of
the International Conference on Machine Learning (ICML-06, Pittsburgh), 2006.
PDF.
17.
J. Schmidhuber and D. Wierstra and F. J. Gomez.
Hybrid Neuroevolution/Regression Search for Sequence Prediction.
Proceedings of the 19th International Joint Conference
on Artificial Intelligence (IJCAI),
2005.
PDF.
16.
D. Wierstra and F. Gomez and J. Schmidhuber.
Modeling systems with internal state using Evolino.
In Proc. of the 2005 conference on genetic and
evolutionary computation (GECCO), Washington, D. C.,
pp. 1795-1802, ACM Press, New York, NY, USA, 2005.
(Got a GECCO best paper award).
PDF.
15.
F. Gomez and J. Schmidhuber.
Co-evolving recurrent neurons learn deep memory POMDPs.
In Proc. of the 2005 conference on genetic and
evolutionary computation (GECCO), Washington, D. C.,
pp. 1795-1802, ACM Press, New York, NY, USA, 2005.
(Nominated for a best paper award).
PDF.
14.
F. J. Gomez and J. Schmidhuber.
Evolving modular fast-weight networks for control.
In W. Duch et al. (Eds.):
Proc. Intl. Conf. on Artificial Neural Networks ICANN'05,
LNCS 3697, pp. 383-389, Springer-Verlag Berlin Heidelberg, 2005.
Featuring a 3-wheeled reinforcement learning robot with distance sensors
that learns without a teacher to balance a jointed pole
indefinitely in a confined 3D environment.
PDF.
13.
A. Graves, S. Fernandez, and J. Schmidhuber.
Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition.
In W. Duch et al. (Eds.):
Proc. Intl. Conf. on Artificial Neural Networks ICANN'05,
LNCS 3697, pp. 799-804, Springer-Verlag Berlin Heidelberg, 2005.
PDF.
12.
N. Beringer and A. Graves and F. Schiel and J. Schmidhuber.
Classifying unprompted speech by retraining LSTM Nets.
In W. Duch et al. (Eds.):
Proc. Intl. Conf. on Artificial Neural Networks ICANN'05,
LNCS 3696, pp. 575-581, Springer-Verlag Berlin Heidelberg, 2005.
PDF.
11.
A. Graves and J. Schmidhuber.
Framewise Phoneme Classification with Bidirectional LSTM Networks.
In Proc. International Joint Conference on Neural Networks
IJCNN'05, 2005.
PDF.
10.
A. Graves, D. Eck and N. Beringer, J. Schmidhuber.
Biologically Plausible Speech Recognition with LSTM Neural Nets.
In J. Ijspeert (Ed.),
First Intl. Workshop on Biologically
Inspired Approaches to Advanced Information Technology,
Bio-ADIT 2004, Lausanne, Switzerland, p. 175-184, 2004.
PDF .
9.
A. Graves, N. Beringer, J. Schmidhuber.
A Comparison Between Spiking and Differentiable Recurrent
Neural Networks on Spoken Digit Recognition.
In Proc. 23rd International Conference on modelling, identification,
and control (IASTED), 2004.
PDF .
8.
B. Bakker, V. Zhumatiy, G. Gruener, and J. Schmidhuber.
A Robot that Reinforcement-Learns to Identify and Memorize Important
Previous Observations
(PDF).
In Proceedings of the 2003 IEEE/RSJ
International Conference on Intelligent Robots and Systems, IROS2003, 2003.
7.
B. Bakker and J. Schmidhuber.
Hierarchical Reinforcement
Learning Based on Subgoal Discovery and Subpolicy Specialization
(PDF).
In F. Groen, N. Amato, A. Bonarini, E. Yoshida, and B. Kröse (Eds.),
Proceedings of the 8-th Conference on Intelligent Autonomous Systems,
IAS-8, Amsterdam, The Netherlands, p. 438-445, 2004.
6.
B. Bakker, F. Linaker, J. Schmidhuber.
Reinforcement Learning in Partially Observable Mobile Robot
Domains Using Unsupervised Event Extraction.
In Proceedings of the 2002
IEEE/RSJ International Conference on
Intelligent Robots and Systems (IROS 2002), Lausanne, 2002.
PDF .
5.
B. Bakker.
Reinforcement Learning with Long Short-Term Memory.
Advances in Neural Information Processing
Systems 13 (NIPS'13), 2002.
(On J. Schmidhuber's CSEM grant 2002.)
4.
D. Eck and J. Schmidhuber.
Learning The Long-Term Structure of the Blues.
In J. Dorronsoro, ed.,
Proceedings of Int. Conf. on Artificial Neural Networks
ICANN'02, Madrid, pages 284-289, Springer, Berlin, 2002.
PDF.
3.
M. Klapper-Rybicka, N. N. Schraudolph, J. Schmidhuber.
Unsupervised Learning in LSTM Recurrent Neural Networks.
In G. Dorffner, H. Bischof, K. Hornik, eds.,
Proceedings of Int. Conf. on Artificial Neural Networks
ICANN'01, Vienna, LNCS 2130, pages 684-691, Springer, 2001.
PDF.
2.
S. Hochreiter and J. Schmidhuber.
LSTM can solve hard long time lag problems.
In M. C. Mozer, M. I. Jordan, T. Petsche, eds.,
Advances in Neural Information Processing Systems 9, NIPS'9,
pages 473-479, MIT Press, Cambridge MA, 1997.
PDF .
HTML.
1.
S. Hochreiter and J. Schmidhuber.
Bridging long time lags by weight guessing and ``Long Short-Term Memory''.
In F. L. Silva, J. C. Principe, L. B. Almeida, eds.,
Frontiers in Artificial Intelligence and Applications, Volume 37,
pages 65-72, IOS Press, Amsterdam, Netherlands, 1996.