Neural networks are particularly well-studied instances of ``generalizers,'' see, e.g., [Maass, 1994,Baum and Haussler, 1989,Amari and Murata, 1993,Wolpert, 1993,Moody, 1992,Pearlmutter and Rosenfeld, 1991,Barron, 1988,Gallant, 1990,Mozer and Smolensky, 1989] and the numerous references given below. For this reason, the simulations presented in this section focus on the task of finding algorithmically simple neural networks with high generalization capability. Let us first briefly look at a few rather recent definitions of ``simplicity'' used in supervised neural net training algorithms. In what follows, ``solutions'' are weight vectors of neural nets.