Jürgen Schmidhuber's work on
Cogbotlab
cylinder with active skin moving through liquid

ARTIFICIAL
EVOLUTION

OVERVIEW
(scroll down for papers)
RNNs control fast weight nets for robot control
Left: our bot with 3 wheels learns to balance two poles with a joint.
Above: our realistic physics simulation of a cylinder with active skin (with 16 actuators) moving through liquid. Goal: minimize drag by learning to create actuator movements that counteract the creation of vortices in the wake. Unprecedented success - see ref 10 (scroll down).

Related work on fluid dynamics: M. Milano, P. Koumoutsakos, J. Schmidhuber. Self-Organizing Nets for Optimization. IEEE Transactions on Neural Networks, 15(3):758-765, 2004. PDF.


SOURCE CODE for some of our novel evolutionary algorithms in our PYBRAIN Machine Learning Library - see video.

6. Compressed Network Search (1995-): Many practical algorithms can evolve hundreds of adaptive parameters, but not millions. Ours can, by evolving compact, compressed descriptions (programs) of huge networks. Compare papers 32a, 32b, 32, 33, 39, 41, 42, 43, 45, 48 below.

5. Natural Evolution Strategies (2009-): A theoretically principled and practical way of evolving solutions based on the natural gradient. See papers 31,34,35,37 below (two best paper awards).

4. Evolino (2005-2008): Evolution for supervised sequence learning - a new class of learning algorithms for supervised recurrent neural networks; often outperforms previous methods. See papers 11,12,15,17,18 below.

3. Co-evolving recurrent neurons or synapses for reinforcement learning (with Faustino Gomez, 2005-2008): Recurrent neural networks (RNNs) are general computers. Evolving their weights means evolving programs. State-of-the-art methods actually use a population of weight vectors for each neuron, and co-evolve all neurons in parallel. Excellent results in various applications. See papers 13,14,16,26 below.

2. Probabilistic incremental program evolution (with Rafal Salustowicz, 1997). Evolving computer programs through probabilistic templates instead of program populations. First approach to evolving entire soccer team strategies from scratch. See papers 3,4,5,6 below.

1. Genetic programming: In 1987, Schmidhuber reinvented GP (apparently first invented by Cramer in 1985) and published the world's 2nd paper on GP. His diploma thesis from the same year described the first approach to Meta-GP (evolving better ways of evolving programs). See papers 1,2 below.

History highlights. Artificial evolution in engineering was pioneered by Rechenberg and his student Schwefel in the 1960s. Through systematic mutations of suboptimal airplane and rocket designs etc they evolved better ones. In the 1970s Holland introduced sexual recombination as an additional means of artificial evolution. (In 1985, Cramer applied such principles to the task of evolving computer programs - Genetic Programming or GP).

GP helix
Evolino for time series prediction
Reinforcement Learning
Pybrain Machine Learning Library for Robot Learning
Learning Robots
Master's Degree in Informatics with a Major in Intelligent Systems -  a master's in computer science, with a specialization in Artificial Intelligence
Related links:

Full publication list
(with additional HTML and pdf links)

Co-evolving recurrent neurons

Evolino

Compressed Network Search

Learning robots

Artificial Intelligence

Metasearching & metalearning & self-improvement

Reinforcement learning (RL)

RL economies

Active exploration and curiosity- driven RL

Hierarchical learning & subgoal generation.

Schmidhuber's CoTeSys group

German home

Fibonacci web design
by J. Schmidhuber

Papers on artificial evolution:

48. J. Koutnik, G. Cuccu, J. Schmidhuber, F. Gomez. Evolving Large-Scale Neural Networks for Vision-Based Reinforcement Learning. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), Amsterdam, 2013. PDF.

47. Yi Sun, F. Gomez, T. Schaul, J. Schmidhuber. A Linear Time Natural Evolution Strategy for Non-Separable Functions. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), Amsterdam, 2013. Preprint arXiv:1106.1998v2 (2011).

46. J. Leitner, S. Harding, M. Frank, A. Förster, J. Schmidhuber. Humanoid Learns to Detect Its Own Hands. IEEE Congress on Evolutionary Computing (CEC), Cancun, Mexico, 2013. PDF.

45. J. Koutnik, G. Cuccu, J. Schmidhuber, F. Gomez. Evolving Large-Scale Neural Networks for Vision-Based TORCS. In Foundations of Digital Games (FDG), Chania, Crete, 2013. PDF.

44. T. Glasmachers, J. Koutnik, J. Schmidhuber. Kernel Representations for Evolving Continuous Functions. Evolutionary Intelligence Journal, 2012. PDF.

43. R. K. Srivastava, F. Gomez, J. Schmidhuber. Generalized Compressed Network Search. In C. Coello Coello, V. Cutello, K. Deb, S. Forrest, G. Nicosia, M. Pavone, eds., 12th Int. Conf. on Parallel Problem Solving from Nature - PPSN XII, Taormina, 2012. PDF.

42. F. Gomez, J. Koutnik, J. Schmidhuber. Compressed Network Complexity Search. In C. Coello Coello, V. Cutello, K. Deb, S. Forrest, G. Nicosia, M. Pavone, eds., 12th Int. Conf. on Parallel Problem Solving from Nature - PPSN XII, Taormina, 2012. Nominated for best paper award. PDF.

41. F. Gomez, J. Koutnik, J. Schmidhuber. Complexity Search for Compressed Neural Networks. Proc. GECCO 2012. PDF.

40. S. Yi, T. Schaul, F. Gomez, J. Schmidhuber. A Linear Time Natural Evolution Strategy for Non-Separable Functions. Proc. GECCO 2012.

39. R. K. Srivastava, J. Schmidhuber, F. Gomez. Generalized Compressed Network Search. Proc. GECCO 2012. PDF.

38. S. Harding, V. Graziano, J. Leitner, J. Schmidhuber. MT-CGP: Mixed Type Cartesian Genetic Programming. Proc. GECCO 2012.

37. T. Schaul, T. Glasmachers, J. Schmidhuber. High Dimensions and Heavy Tails for Natural Evolution Strategies. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2011, Dublin), 2011. PDF.

36. T. Schaul, Yi Sun, D. Wierstra, F. Gomez, J. Schmidhuber. Curiosity-Driven Optimization. IEEE Congress on Evolutionary Computation (CEC-2011), 2011. PDF.

35. T. Glasmachers, T. Schaul, Sun Yi, D. Wierstra, J. Schmidhuber. Exponential Natural Evolution Strategies. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2010), Portland, 2010. PDF. Best paper award.

34. T. Glasmachers, T. Schaul, J. Schmidhuber. A Natural Evolution Strategy for Multi-Objective Optimization. Proceedings of Parallel Problem Solving from Nature (PPSN-2010, Krakow), 2010.

33. J. Koutnik, F. Gomez, J. Schmidhuber (2010). Evolving Neural Networks in Compressed Weight Space. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2010), Portland, 2010. PDF.

32. J. Koutnik, F. Gomez, J. Schmidhuber. Searching for Minimal Neural Networks in Fourier Space. The 3rd Conference on Artificial General Intelligence (AGI-10), 2010. PDF.

32b. J. Schmidhuber. Discovering neural nets with low Kolmogorov complexity and high generalization capability. Neural Networks, 10(5):857-873, 1997. PDF. HTML

32a. J.  Schmidhuber. Discovering solutions with low Kolmogorov complexity and high generalization capability. In A. Prieditis and S. Russell, editors, Machine Learning: Proceedings of the Twelfth International Conference (ICML 1995), pages 488-496. Morgan Kaufmann Publishers, San Francisco, CA, 1995. PDF . HTML.

31. S. Yi, D. Wierstra, T. Schaul, J. Schmidhuber. Efficient Natural Evolution Strategies. Genetic and Evolutionary Computation Conference (GECCO-09), Montreal, 2009. PDF. Best paper award.

30. J. Bayer, D. Wierstra, J. Togelius, J. Schmidhuber. Evolving memory cell structures for sequence learning. Proceedings of the 19th International Conference on Artificial Neural Networks (ICANN-09), Cyprus, 2009. PDF.

29. F. J. Gomez, J. Togelius, J. Schmidhuber. Measuring and Optimizing Behavioral Complexity for Evolutionary Reinforcement Learning . Proceedings of the 19th International Conference on Artificial Neural Networks (ICANN-09), Cyprus, 2009. PDF.

28. N. van Hoorn, J. Togelius, D. Wierstra, J. Schmidhuber. Robust player imitation using multiobjective evolution. Proceedings of the Congress on Evolutionary Computation (CEC-09), Trondheim, 2009. PDF.

27. F. Gomez, J. Schmidhuber, R. Miikkulainen. Accelerated Neural Evolution through Cooperatively Coevolved Synapses. Journal of Machine Learning Research (JMLR), 9:937-965, 2008. PDF.

26. H. Mayer, F. Gomez, D. Wierstra, I. Nagy, A. Knoll, and J. Schmidhuber. A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural Networks. Advanced Robotics, 22/13-14, p. 1521-1537, 2008.

25. J. Togelius, J. Schmidhuber Title: An Experiment in Automatic Game Design Proceedings of the 2008 IEEE Symposium on Computational Intelligence in Games CIG-2008, Perth, Australia, 2008.

24. A. Agapitos, J. Togelius, S. Lucas, J. Schmidhuber Generating Diverse Opponents with Multiobjective Evolution. Proceedings of the 2008 IEEE Symposium on Computational Intelligence in Games CIG-2008, Perth, Australia, 2008.

23. T. Schaul and J. Schmidhuber. A Scalable Neural Network Architecture for Board Games. Proceedings of the 2008 IEEE Symposium on Computational Intelligence in Games CIG-2008, Perth, Australia, 2008. PDF.

22. J. Togelius, T. Schaul, J. Schmidhuber, F. Gomez. Countering Poisonous Inputs with Memetic Neuroevolution. Proceedings of Parallel Problem Solving from Nature PPSN-2008, Dortmund, 2008. PDF.

21. D. Wierstra, T. Schaul, J. Peters, J. Schmidhuber. Fitness Expectation Maximization. Proceedings of Parallel Problem Solving from Nature PPSN-2008, Dortmund, 2008. PDF.

20. D. Wierstra, T. Schaul, J. Peters, J. Schmidhuber. Natural Evolution Strategies. Proceedings of IEEE Congress on Evolutionary Computation CEC-2008, Hongkong, 2008. PDF.

19. J. Togelius, F. Gomez, J. Schmidhuber. Learning What to Ignore: Memetic Climbing in Topology and Weight Space. IEEE WCCI 2008, Hong Kong, 2008. PDF.

18. J. Schmidhuber, D. Wierstra, M. Gagliolo, F. Gomez. Training Recurrent Networks by Evolino. Neural Computation, 19(3): 757-779, 2007. PDF (preprint). Compare Evolino overview.

17. H. Mayer, F. Gomez, D. Wierstra, I. Nagy, A. Knoll, and J. Schmidhuber (2006). A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural Networks. Proceedings of the International Conference on Intelligent Robotics and Systems (IROS-06, Beijing). PDF.

16. F. Gomez, J. Schmidhuber, and R. Miikkulainen (2006). Efficient Non-Linear Control through Neuroevolution. Proceedings of the European Conference on Machine Learning (ECML-06, Berlin). PDF. A new, general method that outperforms many others on difficult control tasks.

15. J. Schmidhuber, M. Gagliolo, D. Wierstra, F. Gomez. Evolino for Recurrent Support Vector Machines. In Proceedings of the European Symposium on Artificial Neural Networks (ESANN-06, Bruge), 2006. Based on TR IDSIA-19-05, v2, 15 Dec 2005. PDF.

14. F. J. Gomez and J. Schmidhuber. Evolving modular fast-weight networks for control. In W. Duch et al. (Eds.): Proc. Intl. Conf. on Artificial Neural Networks ICANN'05, LNCS 3697, pp. 383-389, Springer-Verlag Berlin Heidelberg, 2005. PDF.

13. F. J. Gomez and J. Schmidhuber. Co-evolving recurrent neurons learn deep memory POMDPs. In Proc. of the 2005 conference on genetic and evolutionary computation (GECCO), Washington, D. C., pp. 1795-1802, ACM Press, New York, NY, USA, 2005. Nominated for Best Paper in Coevolution. PDF.

12. J. Schmidhuber and D. Wierstra and F. J. Gomez. Evolino: Hybrid Neuroevolution / Optimal Linear Search for Sequence Learning. Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI), Edinburgh, p. 853-858, 2005. PDF.

11. D. Wierstra and F. Gomez and J. Schmidhuber. Modeling systems with internal state using Evolino. In Proc. of the 2005 conference on genetic and evolutionary computation (GECCO), Washington, D. C., pp. 1795-1802, ACM Press, New York, NY, USA, 2005. Best paper award. PDF.

10. M. Milano, X. Giannakopoulos, P. Koumoutsakos, and J. Schmidhuber. Evolving strategies for active flow control. Congress on Evolutionary Computation, USA, July 2000. PDF.

9. J. Schmidhuber. Sequential decision making based on direct search. In R. Sun and C. L. Giles, eds., Sequence Learning: Paradigms, Algorithms, and Applications. Lecture Notes on AI 1828, p. 203-240, Springer, 2001. PDF . HTML.

8. J. Schmidhuber. Evolutionary Computation vs Reinforcement Learning. Proceedings of 3rd Asia-Pacific Conference on Simulated Evolution and Learning (SEAL2000), Nagoya, Japan, October 2000. PDF. (Keynote speech)

7. J . Schmidhuber. Artificial Curiosity Based on Discovering Novel Algorithmic Predictability Through Coevolution. In P. Angeline, Z. Michalewicz, M. Schoenauer, X. Yao, Z. Zalzala, eds., Congress on Evolutionary Computation, p. 1612-1618, IEEE Press, Piscataway, NJ, 1999.

6. R. Salustowicz and M. Wiering and J. Schmidhuber. Learning team strategies: soccer case studies. Machine Learning, 1999 (127 K).

5. R. Salustowicz and J.  Schmidhuber. From Probabilities to Programs with Probabilistic Incremental Program Evolution. In D. Corne and M. Dorigo and F. Glover, eds., New Ideas in Optimization, p. 433-450, McGraw-Hill, London, 1999.

4. R. Salustowicz and J. Schmidhuber. Probabilistic incremental program evolution. Evolutionary Computation, 5(2):123-141, 1997.

3. R.  Salustowicz and J.  Schmidhuber. Probabilistic incremental program evolution: stochastic search through program space. In van Someren, M., Widmer, G., editors, Machine Learning: ECML-97, Lecture Notes in Artificial Intelligence 1224, pages 213-220, Springer, 1997.

2. J.  Schmidhuber. Evolutionary principles in self-referential learning, or on learning how to learn: The meta-meta-... hook. Diploma thesis, Institut für Informatik, Technische Universität München, 1987. HTML.

1. D. Dickmanns, J. Schmidhuber, and A. Winklhofer. Der genetische Algorithmus: Eine Implementierung in Prolog. Fortgeschrittenenpraktikum, Institut für Informatik, Lehrstuhl Prof. Radig, Technische Universität München, 1987. HTML.


2011: First Superhuman Visual Pattern Recognition
My Deep Learning since 1991
Goedel machine