We compare LOCOCODE to
``independent component analysis'' (ICA, e.g.,
[5,1,4,19])
and ``principal component analysis'' (PCA, e.g., [21]).
ICA is realized by Cardoso's
JADE algorithm, which
is based on whitening and subsequent
joint diagonalization of 4th-order cumulant matrices.
To measure the information conveyed by resulting codes
we train a standard backprop net on the training set used for
code generation. Its inputs are the code components; its task is
to reconstruct the original input. The test set consists of 500
off-training set exemplars (in the case of real world images we use
a separate test image). *Coding efficiency*
is the average number of bits needed to code a test
set input pixel. The code components are scaled to the interval
and partitioned into discrete intervals.
Assuming independence of the code components
we estimate the probability of
each discrete code value by Monte Carlo sampling on the training set.
To obtain the test set codes' bits per pixel (Shannon's optimal value)
the average sum of all negative logarithms of code component
probabilities is divided by the number of input components.
All details necessary for reimplementation
are given in [15].

**Noisy bars adapted from [11,12].**
The input is a pixel grid
with horizontal and vertical
bars at random positions.
The task is to extract the independent features (the bars).
Each of the 10 possible bars
appears with probability .
In contrast to
[11,12]
we allow for bar type mixing -- *this makes the task harder.*
Bar intensities vary in ;
input units that see a pixel of a bar are activated correspondingly
others adopt activation .
We add Gaussian noise with variance 0.05 and mean 0
to each pixel.
For ICA and PCA we have to provide information
about the number (ten) of independent sources
(tests of versions with assumed sources will
be denoted by ICA- and PCA-).
LOCOCODE does not require this
-- using 25 hidden units (HUs) we expect
LOCOCODE to *prune* the 15 superfluous HUs.

**Results.**
See Table 1.
While the reconstruction errors of all methods
are similar, LOCOCODE has the best coding efficiency.
15 of the 25 HUs are indeed automatically pruned:
LOCOCODE finds an
optimal factorial code which exactly mirrors
the pattern generation process.
PCA codes and ICA-15 codes, however, are unstructured and dense.
While ICA-10 codes are almost sparse and
do recognize some sources, the sources are not clearly separated
like with LOCOCODE
--
compare the weight patterns
shown in [15].

**Real world images.**
Now we use more realistic input data, namely
subsections of: 1) the aerial shot of a village, 2) an image
of wood cells, and
3) an image of striped piece of wood.
Each image has
pixels,
each taking on one of 256 gray levels.
( for village)
pixels subsections are randomly
chosen as training inputs.
Test sets stem from images similar
to 1), 2), and 3).

**Results.**
For the village image
LOCOCODE discovers
on-center-off-surround hidden units
forming a sparse code.
For the other two images LOCOCODE also finds
appropriate feature
detectors -- see weight patterns shown
in [15].
Using its compact, low-complexity features
it always codes more efficiently than ICA and PCA.