next up previous
Next: ACKNOWLEDGMENTS Up: SEMILINEAR PREDICTABILITY MINIMIZATION PRODUCES Previous: APPLICATION: IMAGE PROCESSING

DISCUSSION

We do not claim that PM is the only parallel method (as opposed to sequential methods, e.g. Rubner and Schulten, 1990) that can lead to well-known feature detectors. For instance, in case of Gaussian input distributions, Linsker's linear approach [Linsker, 1986b,Linsker, 1986a] for single output units also generates certain kinds of orientation sensitive fields (see also MacKay and Miller, 1990). This holds for more structured input data as well (Linsker, personal communication, 1994). In case of multiple code units, however, to prevent different code units from representing the same information, Linsker's INFOMAX approach (1988) requires to compute the derivatives of determinants of covariance matrices, which is computationally expensive, and also biologically implausible (the multiple cell approach presented in Linsker (1986b) does not have certain problems of his later infomax approach, but it also does not have that nice theoretical foundation). Finally, it is conceivable that Földiák's system (1990), Rubner and Tavan's system (1989), and Deco and Obradovic's system (1996), might come up with similar edge detectors when applied to real world images. Unlike these approaches (and unlike other similar systems), however, our feedforward net does neither require time consuming settling phases (due to recurrent ``anti-Hebbian'' connections for lateral inhibition) nor analytic computation of the weight vectors.

Future research. We implemented a hierarchy of processing stages, each consisting of code modules and predictors as above. Each stage computes the input to the next stage. Preliminary tests again led to feature detectors causing high information throughput. However, unlike receptive fields of feature detectors observed in the first layer, receptive fields in higher layers appeared rather complex and did not exhibit any obvious structure. We would like to test the system on large data sets of real world scenes. We expect that this will lead to successively more complex and more specialized feature detectors, hopefully not only potentially useful for technical applications but also qualitatively related to those observed in biological systems. Another interesting experiment will be to add neighborhood relationships between the code units, to see whether this leads to automatic development of a smoothly-varying map of orientation preference. Unfortunately, however, our current hardware equipment does not permit large scale applications.


next up previous
Next: ACKNOWLEDGMENTS Up: SEMILINEAR PREDICTABILITY MINIMIZATION PRODUCES Previous: APPLICATION: IMAGE PROCESSING
Juergen Schmidhuber 2003-02-17


Back to Independent Component Analysis page.