06-July-2014: Giving a tutorial titled "Slow Feature Analysis for Curiosity-Driven Agents" at the IEEE WCCI-2014, Beijing, China.
Curiosity Driven Modular Incremental Slow Feature Analysis
(Curious Dr. MISFA)
In the absence of external guidance, how can an agent learn to solve increasingly complex tasks from an abundance of sensory information? It needs to be self-motivated (curious) and able to continually build compact representations of sensory inputs to encode different aspects of the changing environment. In the case of low-dimensional inputs from simple environments, previous implementations of curiosity based reinforcement learning (CDRL) enabled learning skills by rewarding improvements of an internal world model. Here, however, we consider high-dimensional observations from complex environments. An online unsupervised learning algorithm called Incremental Slow Feature Analysis (IncSFA) is used to build lower-dimensional but informative input representations. As the input distribution changes through the learner's actions however, IncSFA gradually forgets previously learned representations. To address this, we introduce an multi-module IncSFA to create a modular curiosity-driven system. In order of increasing learning difficulty, multiple abstract slow-feature representations are learned autonomously, while CDRL learns the skills needed to generate them. We prove stability of our algorithm under mild conditions, and experimentally demonstrate its performance.
J3. M. Luciw*, V. R. Kompella*, S. Kazerounian and J. Schmidhuber. "An intrinsic value system for developing multiple invariant representations with incremental slowness learning", Frontiers in Neurorobotics, Vol. 7 (9), 2013. *Joint first authors. Link to paper.
C8. V. R. Kompella, M. Stollenga, M. Luciw and J. Schmidhuber. "Explore to See, Learn to Perceive, Get the Actions for Free: SKILLABILITY" , To Appear in the Proc. of IEEE International Joint Conference on Neural Networks (IJCNN), Beijing, 2014. Link to paper.
C6. V. R. Kompella, M. Luciw, M. Stollenga, L. Pape and J. Schmidhuber. "Autonomous Learning of Abstractions using Curiosity-Driven Modular Incremental Slow Feature Analysis. (Curious Dr. MISFA)" , IEEE International Conference on Developmental and Learning and Epigenetic Robotics (ICDL-EpiRob), San Diego, 2012. Link to paper.