This section presents another alternative way of ``predicting away'' redundant information in sequences. Again, we pre-process input sequences by a network that tries to predict the next input, given previous inputs. The input vector corresponding to time step of sequence is denoted by . The networks real-valued output vector is denoted by . Among the possible input vectors, there is one with minimal Euclidean distance to . This one is denoted by . is interpreted as the deterministic vector-valued prediction of .

It is important to observe that
all information about the input vector (at time ) is
conveyed by the following data:
the time ,
a description of the predictor
and its initial state,
and the set

In what follows, this observation will be used to compress text files.

Juergen Schmidhuber 2003-02-13