The methods described in this paper are primarily devoted to
finding binary or at least *quasi-binary* codes.
Each code symbol participating in a quasi-binary code
is either 0 or 1 in response to a given input pattern or
emits a constant value in response to every input pattern.
Therefore, binary codes are a special case of quasi-binary codes.
Most of our quasi-binary codes will be created by
starting out from real-valued codes.

Recall that there are three criteria that a binary factorial code must fulfill:

1. *The binary criterion*: Each code-symbol should be either
1 or 0 in response to a given input pattern.

2. *The invertibility criterion*: It must be possible to reconstruct the
input from the code.
In cases where the environment is too complex (or too noisy) to be
fully coded into limited internal representations (i.e.,
in the case of binary codes
where there are more than input patterns),
we want to relax the invertibility criterion. In that case,
we still want the internal representations to convey maximal
information about the inputs. The focus of this paper, however,
is on situations like the ones studied in (Barlow et. al, 1989):
Noise-free environments and sufficient representational
capacity in the representational units.
In the latter case, reversibility is equivalent to Infomax
*à la* Linsker (1988).

3. *The independence criterion*: The occurrence of each code symbol
ought to be independent of all other code symbols. If the
binary criterion is fulfilled, then
we may rewrite
the independence criterion by requiring that

The latter condition implies that does not depend on . In other words, is computable from a constant. Note that with

Back to Independent Component Analysis page.