Next: EXPERIMENT 4: vowel recognition
Up: EXPERIMENT 3: More Realistic
Previous: EXPERIMENT 3: More Realistic
Table 2 shows that most lococodes and some ICA codes are sparse,
while most PCA codes are dense. Assuming that each visual input
consists of many components collectively describable by few input features,
LOCOCODE seems preferable.
Table 2:
Overview over experiments 2 and 3:
name of experiment, input field size, coding method,
number of relevant code components (code size),
reconstruction error, nature of code observed on the test set.
PCA's and ICA's code sizes are prewired. LOCOCODE's,
however, are found automatically.
The final column shows
coding efficiency measured in bits per pixels (for code components
mapped to 100 discrete intervals) and reconstruction error (for
this discrete code).
LOCOCODE exhibits superior coding efficiency.
Exp. |
input |
method |
# code |
reconst. |
code type |
code efficency |
|
field |
|
comp. |
error |
|
- reconst. |
bars |
|
LOC |
10 |
0.08 |
sparse (factorial) |
1.22 - 0.09 |
bars |
|
ICA |
10 |
0.08 |
almost sparse |
1.44 - 0.09 |
bars |
|
PCA |
10 |
0.09 |
dense |
1.43 - 0.09 |
bars |
|
ICA |
15 |
0.09 |
dense |
2.19 - 0.10 |
bars |
|
PCA |
15 |
0.16 |
dense |
2.06 - 0.16 |
noisy bars |
|
LOC |
10 |
1.05 |
sparse (factorial) |
1.37 - 1.06 |
noisy bars |
|
ICA |
10 |
1.02 |
almost sparse |
1.68 - 1.03 |
noisy bars |
|
PCA |
10 |
1.03 |
dense |
1.66 - 1.04 |
noisy bars |
|
ICA |
15 |
0.71 |
dense |
2.50 - 0.73 |
noisy bars |
|
PCA |
15 |
0.72 |
dense |
2.47 - 0.72 |
village image |
|
LOC |
10 |
8.29 |
sparse |
0.69 - 8.29 |
village image |
|
ICA |
10 |
7.90 |
dense |
0.80 - 7.91 |
village image |
|
PCA |
10 |
9.21 |
dense |
0.80 - 9.22 |
village image |
|
ICA |
15 |
6.57 |
dense |
1.20 - 6.58 |
village image |
|
PCA |
15 |
8.03 |
dense |
1.19 - 8.04 |
wood cell image |
|
LOC |
11 |
0.84 |
sparse |
0.96 - 0.86 |
wood cell image |
|
ICA |
11 |
0.87 |
sparse |
0.98 - 0.89 |
wood cell image |
|
PCA |
11 |
0.72 |
almost sparse |
0.96 - 0.73 |
wood cell image |
|
ICA |
15 |
0.36 |
sparse |
1.32 - 0.39 |
wood cell image |
|
PCA |
15 |
0.33 |
dense |
1.28 - 0.34 |
wood piece image |
|
LOC |
4 |
0.83 |
almost sparse |
0.39 - 0.84 |
wood piece image |
|
ICA |
4 |
0.86 |
almost sparse |
0.40 - 0.87 |
wood piece image |
|
PCA |
4 |
0.83 |
almost sparse |
0.40 - 0.84 |
wood piece image |
|
ICA |
10 |
0.72 |
almost sparse |
1.00 - 0.76 |
wood piece image |
|
PCA |
10 |
0.53 |
almost sparse |
0.91 - 0.54 |
|
Conclusion.
Unlike standard BP-trained AAs, FMS-trained AAs
generate highly structured sensory codes.
FMS automatically prunes superfluous units.
PCA experiments indicate that the remaining code units
suit the various coding tasks well.
Taking into account statistical properties of the visual
input data, LOCOCODE
generates appropriate feature detectors such as the
familiar on-center-off-surround and bar detectors.
It also produces biologically plausible sparse codes (standard AAs do
not).
FMS's objective function, however, does not contain explicit terms
enforcing such codes (this contrasts previous methods, e.g.,
Olshausen and Field 1996).
The experiments show that equally-sized PCA codes, ICA codes, and lococodes
convey approximately the same
information.
LOCOCODE, however, codes with fewer bits per pixel.
Unlike PCA and ICA, it determines the code size automatically.
Some of the feature detectors obtained by LOCOCODE are similar
to those found by ICA. In cases where we know the true
input causes, however,
LOCOCODE does find them whereas ICA does not.
Next: EXPERIMENT 4: vowel recognition
Up: EXPERIMENT 3: More Realistic
Previous: EXPERIMENT 3: More Realistic
Juergen Schmidhuber
2003-02-13
Back to Independent Component Analysis page.