The statistician's point of view is this: Suppose we draw input patterns from an -dimensional distribution given by a random variable . The -th pattern is a vector .

Redundancy means
that pattern components share
mutual information. It means that
if we know
certain components of some pattern,
then we already know something about
other components. Redundancy implies
statistical dependence of the components:

Juergen Schmidhuber 2003-02-19