06-July-2014: Giving a tutorial titled "Slow Feature Analysis for Curiosity-Driven Agents" at the IEEE WCCI-2014, Beijing, China.
Incremental Slow Feature Analysis (IncSFA)
Figure 1: A toy example: (a) Consider a zero-mean input signal (input points represented by black dots) that spatially resembles white noise. Linear spatial feature extractors (such as PCA) will not prefer any direction over any other. (b) Certain hidden directions can be uncovered if the changes between subsequent time instants are used. Here, the arrows show a short representative sequence of input. (c) Space of all difference vectors between consecutive time instants. The direction of the least variance is the direction of slowest change (first Slow Feature).
The Slow Feature Analysis (SFA) unsupervised learning framework extracts
features representing the underlying causes of the changes within a temporally
coherent high-dimensional raw sensory input signal. We develop the first online version of
SFA, via a combination of incremental Principal Components Analysis and Minor Components Analysis.
Unlike standard batch-based SFA, online SFA adapts along with non-stationary environments,
which makes it a generally useful unsupervised preprocessor for autonomous learning agents.
We compare online SFA to batch SFA in several experiments and show that it indeed learns without
a teacher to encode the input stream by informative slow features representing meaningful abstract
environmental properties. We extend online SFA to deep networks in hierarchical fashion, and use
them to successfully extract abstract object position information from high-dimensional video.
J2. V. R. Kompella, M. Luciw and J. Schmidhuber. "Incremental Slow Feature Analysis: Adaptive Low-Complexity Slow Feature Updating from High-Dimensional Input Streams", Neural Computation Journal, Vol. 24 (11), pp. 2994--3024, 2012. Link to preprint.
C4. V. R. Kompella, M. Luciw and J. Schmidhuber. "Incremental Slow Feature Analysis", 22nd International Joint Conference on Artificial Intelligence (IJCAI), Barcelona, 2011. Link to paper.
W1. M. Luciw, V. R. Kompella and J. Schmidhuber. "Hierarchical Incremental Slow Feature Analysis" , Workshop on Deep Hierarchies in Vision (DHV, Vienna), 2012. Link to poster.
Videos: Video I