INCREMENTAL LEARNING

With many typical ``incremental'' learning situations in the real world, there is more informative feedback than with the tasks above, where there is none. The original universal search procedure as formulated by Levin is not designed for optimal use of error feedback in ``incremental'' learning. However, there appears to be more than one reasonable way of appropriately extending universal search. Some ideas are given in Solomonoff's (1986) and Paul's (1991) more recent theoretical work. Others are presented in (Schmidhuber, 1994a), where mutations of previously useful programs are listed in order of their Levin complexities, until additional improvements are found. (Schmidhuber, 1994a) also presents the first experimental results. They show that ``incremental'' extensions can allow for much faster learning but tend to find less elegant programs.

Ongoing research. Very recently, and for the first time, incremental learning in general environments was put on a basis that appears theoretically sound: Schmidhuber (1994b) goes beyond the current paper, by presenting a novel machine learning paradigm called the ``incremental self-improvement paradigm''. In principle, a probabilistic system based on this paradigm is able to use previous experience to improve itself, and to improve the way it improves itself, etc. Essentially, the system uses previous experience to learn to modify context-dependent primitive probabilities in a way that leads to more success per time interval, thus learning to make better and better use of its computational resources. The basic ideas are briefly described in the following, concluding subsection.

Back to Optimal Universal Search page
Back to Program Evolution page
Back to Algorithmic Information page
Back to Speed Prior page