... LEARNING1
IN X. YAO, EDITOR, EVOLUTIONARY COMPUTATION: THEORY AND APPLICATIONS. CHAPTER 3, PP.81-123, SCIENTIFIC PUBL. CO., SINGAPORE, 1999.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
... actions2
Leslie Kaelbling sometimes refers to this as ``writing on the walls'' but says that the ``real'' name is ``stigmergy'' (personal communication, 1994/1995).
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
... ``true''3
I am not talking about fixed learning algorithms for adjusting the parameters of others. For instance, GAs are sometimes used to adjust learning rates of gradient based neural nets, etc. Or a neural net is used to compute the weights of another neural net. In the literature, one can find quite a few approaches of this kind (too many to cite them all -- I settle by citing none, not even my own). Although such approaches sometimes may have their merits, they do not deserve the attribute ``self-referential'' -- the additional level typically just defers the credit assignment problem. However, there were a few apparently more general approaches. For instance, Lenat [17] reports that his EURISKO system was able to discover certain heuristics for discovering heuristics. His approach, however, as well as all other previous approaches I am aware of, were either quite limited (many essential aspects of system behavior being unmodifiable), and/or lacked a sound, convincing global credit assignment strategy (as embodied by the SSA pushing and popping processes).
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
... experience4
Solomonoff appears to be well aware of problems with the meta-version: at the end of his 1990 paper, he refers to self-improvement as a ``more distant goal'': ``The kind of training needed involves more mathematics and work on various kinds of optimization problems -- ultimately problems of improving computer programs.'' Another ``more distant goal'' mentioned by Solomonoff is to let the system work ``on an unordered batch of problems -- deciding itself which are the easiest, and solving them first''. Note that SSA addresses both goals, without depending on a meta-version of universal search.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.