next up previous


Barlow et al., 1989
Barlow, H. B., Kaushal, T. P., and Mitchison, G. J. (1989).
Finding minimum entropy codes.
Neural Computation, 1(3):412-423.

Becker and Hinton, 1989
Becker, S. and Hinton, G. E. (1989).
Spatial coherence as an internal teacher for a neural network.
Technical Report CRG-TR-89-7, Department of Computer Science, University of Toronto, Ontario.

Bridle and MacKay, 1992
Bridle, J. S. and MacKay, D. J. C. (1992).
Unsupervised classifiers, mutual information and `phantom' targets.
In Lippman, D. S., Moody, J. E., and Touretzky, D. S., editors, Advances in Neural Information Processing Systems 4, to appear. San Mateo, CA: Morgan Kaufmann.

LeCun, 1985
LeCun, Y. (1985).
Une procédure d'apprentissage pour réseau à seuil asymétrique.
Proceedings of Cognitiva 85, Paris, pages 599-604.

Linsker, 1988
Linsker, R. (1988).
Self-organization in a perceptual network.
IEEE Computer, 21:105-117.

Nowlan, 1988
Nowlan, S. J. (1988).
Auto-encoding with entropy constraints.
In Proceedings of INNS First Annual Meeting, Boston, MA.
Also published in special supplement to Neural Networks.

Parker, 1985
Parker, D. B. (1985).
Technical Report TR-47, Center for Comp. Research in Economics and Management Sci., MIT.

Prelinger, 1992
Prelinger, D. (1992).
Diploma thesis.
Institut für Informatik, Technische Universität München.

Rumelhart et al., 1986
Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986).
Learning internal representations by error propagation.
In Parallel Distributed Processing, volume 1, pages 318-362. MIT Press.

Schmidhuber, 1992
Schmidhuber, J. H. (1992).
Learning factorial codes by predictability minimization.
Neural Computation, 4(6):863-879.

Schmidhuber and Prelinger, 1992
Schmidhuber, J. H. and Prelinger, D. (1992).
Discovering predictable classifications.
Technical Report CU-CS-626-92, Dept. of Comp. Sci., University of Colorado at Boulder.

Shannon, 1948
Shannon, C. E. (1948).
A mathematical theory of communication (parts I and II).
Bell System Technical Journal, XXVII:379-423.

Werbos, 1974
Werbos, P. J. (1974).
Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences.
PhD thesis, Harvard University.

Zemel and Hinton, 1991
Zemel, R. S. and Hinton, G. E. (1991).
Discovering viewpoint-invariant relationships that characterize objects.
In Lippman, D. S., Moody, J. E., and Touretzky, D. S., editors, Advances in Neural Information Processing Systems 3, pages 299-305. San Mateo, CA: Morgan Kaufmann.

Figure 1: Two networks try to transform their different inputs to obtain the same representation. Each network is encouraged to tell something about its input by means of the recent technique for `predictability minimization'. This technique requires additional intra-representational predictors (8 of them shown above) for detecting redundancies among the output units of the networks. Alternatives are provided in the text.

Juergen Schmidhuber 2003-02-13