Bibliography

Barlow et al., 1989

Barlow, H. B., Kaushal, T. P., and Mitchison, G. J. (1989).
Finding minimum entropy codes.
Neural Computation, 1(3):412-423.

Becker and Hinton, 1989

Becker, S. and Hinton, G. E. (1989).
Spatial coherence as an internal teacher for a neural network.
Technical Report CRG-TR-89-7, Department of Computer Science, University of Toronto, Ontario.

Bridle and MacKay, 1992

Bridle, J. S. and MacKay, D. J. C. (1992).
Unsupervised classifiers, mutual information and `phantom' targets.
In Lippman, D. S., Moody, J. E., and Touretzky, D. S., editors, Advances in Neural Information Processing Systems 4, to appear. San Mateo, CA: Morgan Kaufmann.

LeCun, 1985

LeCun, Y. (1985).
Une procédure d'apprentissage pour réseau à seuil asymétrique.
Proceedings of Cognitiva 85, Paris, pages 599-604.

Linsker, 1988

Linsker, R. (1988).
Self-organization in a perceptual network.
IEEE Computer, 21:105-117.

Nowlan, 1988

Nowlan, S. J. (1988).
Auto-encoding with entropy constraints.
In Proceedings of INNS First Annual Meeting, Boston, MA.
Also published in special supplement to Neural Networks.

Parker, 1985

Parker, D. B. (1985).
Learning-logic.
Technical Report TR-47, Center for Comp. Research in Economics and Management Sci., MIT.

Prelinger, 1992

Prelinger, D. (1992).
Diploma thesis.
Institut für Informatik, Technische Universität München.

Rumelhart et al., 1986

Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986).
Learning internal representations by error propagation.
In Parallel Distributed Processing, volume 1, pages 318-362. MIT Press.

Schmidhuber, 1992

Schmidhuber, J. H. (1992).
Learning factorial codes by predictability minimization.
Neural Computation, 4(6):863-879.

Schmidhuber and Prelinger, 1992

Schmidhuber, J. H. and Prelinger, D. (1992).
Discovering predictable classifications.
Technical Report CU-CS-626-92, Dept. of Comp. Sci., University of Colorado at Boulder.

Shannon, 1948

Shannon, C. E. (1948).
A mathematical theory of communication (parts I and II).
Bell System Technical Journal, XXVII:379-423.

Werbos, 1974

Werbos, P. J. (1974).
Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences.
PhD thesis, Harvard University.

Zemel and Hinton, 1991

Zemel, R. S. and Hinton, G. E. (1991).
Discovering viewpoint-invariant relationships that characterize objects.
In Lippman, D. S., Moody, J. E., and Touretzky, D. S., editors, Advances in Neural Information Processing Systems 3, pages 299-305. San Mateo, CA: Morgan Kaufmann.

Figure 1: Two networks try to transform their different inputs to obtain the same representation. Each network is encouraged to tell something about its input by means of the recent technique for `predictability minimization'. This technique requires additional intra-representational predictors (8 of them shown above) for detecting redundancies among the output units of the networks. Alternatives are provided in the text.