Next: ACKNOWLEDGMENTS
Up: SEMILINEAR PREDICTABILITY MINIMIZATION PRODUCES
Previous: APPLICATION: IMAGE PROCESSING
We do not claim that PM is the only
parallel method
(as opposed to sequential methods, e.g.
Rubner and Schulten, 1990)
that can lead to well-known feature detectors.
For instance, in case of Gaussian
input distributions,
Linsker's linear approach
[Linsker, 1986b,Linsker, 1986a]
for single output units
also generates
certain kinds of orientation sensitive fields
(see also MacKay and Miller, 1990).
This holds for more structured input data as well (Linsker,
personal communication, 1994).
In case of multiple code units, however,
to prevent different code units from representing the same
information, Linsker's INFOMAX approach (1988)
requires to compute the derivatives of determinants of covariance
matrices, which is computationally expensive, and also
biologically implausible
(the multiple cell approach presented in Linsker (1986b) does
not have certain
problems of his later infomax approach, but it also does not
have that nice theoretical foundation).
Finally, it is conceivable that
Földiák's system (1990),
Rubner and Tavan's system (1989),
and Deco and Obradovic's system (1996),
might come up with similar edge detectors when
applied to real world images.
Unlike these approaches (and unlike other similar systems),
however, our feedforward net does neither
require time consuming settling phases (due to recurrent
``anti-Hebbian'' connections for lateral inhibition) nor
analytic computation of the weight vectors.
Future research.
We implemented a hierarchy of processing stages, each consisting
of code modules and predictors as above.
Each stage computes the input to the next stage.
Preliminary tests again led to feature detectors
causing high information throughput.
However,
unlike receptive fields of feature
detectors observed in the first layer,
receptive fields in higher layers
appeared rather complex and did not exhibit any obvious
structure. We would like to test the system
on large data sets of real world scenes.
We expect that this
will lead to successively more complex and more specialized
feature detectors, hopefully not only potentially useful
for technical applications but also qualitatively
related to those observed in biological systems.
Another interesting experiment will be to add neighborhood relationships
between the code units, to see whether this leads to automatic
development of a smoothly-varying map of orientation preference.
Unfortunately, however, our current hardware equipment does
not permit large scale applications.
Next: ACKNOWLEDGMENTS
Up: SEMILINEAR PREDICTABILITY MINIMIZATION PRODUCES
Previous: APPLICATION: IMAGE PROCESSING
Juergen Schmidhuber
2003-02-17
Back to Independent Component Analysis page.