next up previous
Next: More Formally Up: The New AI: General Previous: The New AI: General


Remarkably, there is a theoretically optimal way of making predictions based on observations, rooted in the early work of Solomonoff and Kolmogorov [62,28]. The approach reflects basic principles of Occam's razor: simple explanations of data are preferable to complex ones.

The theory of universal inductive inference quantifies what simplicity really means. Given certain very broad computability assumptions, it provides techniques for making optimally reliable statements about future events, given the past.

Once there is an optimal, formally describable way of predicting the future, we should be able to construct a machine that continually computes and executes action sequences that maximize expected or predicted reward, thus solving an ancient goal of AI research.

For many decades, however, AI researchers have not paid a lot of attention to the theory of inductive inference. Why not? There is another reason besides the fact that most of them have traditionally ignored theoretical computer science: the theory has been perceived as being associated with excessive computational costs. In fact, its most general statements refer to methods that are optimal (in a certain asymptotic sense) but incomputable. So researchers in machine learning and artificial intelligence have often resorted to alternative methods that lack a strong theoretical foundation but at least seem feasible in certain limited contexts. For example, since the early attempts at building a ``General Problem Solver'' [36,43] much work has been done to develop mostly heuristic machine learning algorithms that solve new problems based on experience with previous problems. Many pointers to learning by chunking, learning by macros, hierarchical learning, learning by analogy, etc. can be found in Mitchell's book [34] and Kaelbling's survey [27].

Recent years, however, have brought substantial progress in the field of computable and feasible variants of optimal algorithms for prediction, search, inductive inference, problem solving, decision making, and reinforcement learning in very general environments. In what follows I will focus on the results obtained at IDSIA.

Sections 3, 4, 7 relate Occam's razor and the notion of simplicity to the shortest algorithms for computing computable objects, and will concentrate on recent asymptotic optimality results for universal learning machines, essentially ignoring issues of practical feasibility--compare Hutter's contribution [25] in this volume.

Section 5, however, will focus on our recent non-traditional simplicity measure which is not based on the shortest but on the fastest way of describing objects, and Section 6 will use this measure to derive non-traditional predictions concerning the future of our universe.

Sections 8, 9, 10 will finally address quite pragmatic issues and ``true'' time-optimality: given a problem and only so much limited computation time, what is the best way of spending it on evaluating solution candidates? In particular, Section 9 will outline a bias-optimal way of incrementally solving each task in a sequence of tasks with quickly verifiable solutions, given a probability distribution (the bias) on programs computing solution candidates. Bias shifts are computed by program prefixes that modify the distribution on their suffixes by reusing successful code for previous tasks (stored in non-modifiable memory). No tested program gets more runtime than its probability times the total search time. In illustrative experiments, ours becomes the first general system to learn a universal solver for arbitrary $n$ disk Towers of Hanoi tasks (minimal solution size $2^n-1$). It demonstrates the advantages of incremental learning by profiting from previously solved, simpler tasks involving samples of a simple context free language. Sections 10 discusses how to use this approach for building general reinforcement learners.

Finally, Section 11 will summarize the recent Gödel machine [56], a self-referential, theoretically optimal self-improver which explicitly addresses the `Grand Problem of Artificial Intelligence' [58] by optimally dealing with limited resources in general reinforcement learning settings.

next up previous
Next: More Formally Up: The New AI: General Previous: The New AI: General
Juergen Schmidhuber 2003-11-27