next up previous
Next: EXPERIMENT 4: vowel recognition Up: EXPERIMENT 3: More Realistic Previous: EXPERIMENT 3: More Realistic

Overview over experiments 2 and 3

Table 2 shows that most lococodes and some ICA codes are sparse, while most PCA codes are dense. Assuming that each visual input consists of many components collectively describable by few input features, LOCOCODE seems preferable.


Table 2: Overview over experiments 2 and 3: name of experiment, input field size, coding method, number of relevant code components (code size), reconstruction error, nature of code observed on the test set. PCA's and ICA's code sizes are prewired. LOCOCODE's, however, are found automatically. The final column shows coding efficiency measured in bits per pixels (for code components mapped to 100 discrete intervals) and reconstruction error (for this discrete code). LOCOCODE exhibits superior coding efficiency.
Exp. input method # code reconst. code type code efficency
  field   comp. error   - reconst.
bars $5 \times 5$ LOC 10 0.08 sparse (factorial) 1.22 - 0.09
bars $5 \times 5$ ICA 10 0.08 almost sparse 1.44 - 0.09
bars $5 \times 5$ PCA 10 0.09 dense 1.43 - 0.09
bars $5 \times 5$ ICA 15 0.09 dense 2.19 - 0.10
bars $5 \times 5$ PCA 15 0.16 dense 2.06 - 0.16
noisy bars $5 \times 5$ LOC 10 1.05 sparse (factorial) 1.37 - 1.06
noisy bars $5 \times 5$ ICA 10 1.02 almost sparse 1.68 - 1.03
noisy bars $5 \times 5$ PCA 10 1.03 dense 1.66 - 1.04
noisy bars $5 \times 5$ ICA 15 0.71 dense 2.50 - 0.73
noisy bars $5 \times 5$ PCA 15 0.72 dense 2.47 - 0.72
village image $7 \times 7$ LOC 10 8.29 sparse 0.69 - 8.29
village image $7 \times 7$ ICA 10 7.90 dense 0.80 - 7.91
village image $7 \times 7$ PCA 10 9.21 dense 0.80 - 9.22
village image $7 \times 7$ ICA 15 6.57 dense 1.20 - 6.58
village image $7 \times 7$ PCA 15 8.03 dense 1.19 - 8.04
wood cell image $7 \times 7$ LOC 11 0.84 sparse 0.96 - 0.86
wood cell image $7 \times 7$ ICA 11 0.87 sparse 0.98 - 0.89
wood cell image $7 \times 7$ PCA 11 0.72 almost sparse 0.96 - 0.73
wood cell image $7 \times 7$ ICA 15 0.36 sparse 1.32 - 0.39
wood cell image $7 \times 7$ PCA 15 0.33 dense 1.28 - 0.34
wood piece image $7 \times 7$ LOC 4 0.83 almost sparse 0.39 - 0.84
wood piece image $7 \times 7$ ICA 4 0.86 almost sparse 0.40 - 0.87
wood piece image $7 \times 7$ PCA 4 0.83 almost sparse 0.40 - 0.84
wood piece image $7 \times 7$ ICA 10 0.72 almost sparse 1.00 - 0.76
wood piece image $7 \times 7$ PCA 10 0.53 almost sparse 0.91 - 0.54


Conclusion. Unlike standard BP-trained AAs, FMS-trained AAs generate highly structured sensory codes. FMS automatically prunes superfluous units. PCA experiments indicate that the remaining code units suit the various coding tasks well. Taking into account statistical properties of the visual input data, LOCOCODE generates appropriate feature detectors such as the familiar on-center-off-surround and bar detectors. It also produces biologically plausible sparse codes (standard AAs do not). FMS's objective function, however, does not contain explicit terms enforcing such codes (this contrasts previous methods, e.g., Olshausen and Field 1996).

The experiments show that equally-sized PCA codes, ICA codes, and lococodes convey approximately the same information. LOCOCODE, however, codes with fewer bits per pixel. Unlike PCA and ICA, it determines the code size automatically. Some of the feature detectors obtained by LOCOCODE are similar to those found by ICA. In cases where we know the true input causes, however, LOCOCODE does find them whereas ICA does not.


next up previous
Next: EXPERIMENT 4: vowel recognition Up: EXPERIMENT 3: More Realistic Previous: EXPERIMENT 3: More Realistic
Juergen Schmidhuber 2003-02-13


Back to Independent Component Analysis page.