Let us define
(5) |
Like in section 4.6 we drop the global invertibility term and
redefine the total objective function to be maximized
by the representational modules as
(6) |
Conjecture. I conjecture that if there exists a quasi-binary factorial code for a given pattern ensemble, then among all possible (real-valued or binary) codes is maximized with a quasi-binary factorial code, even if .
If this conjecture is true, then we may forget about the -term in (9) and simply write . In this case, all representational units simply try to maximize the same function that the predictors try to minimize, namely, . In other words, this generates a symmetry between two forces that fight each other - one trying to predict, the other one trying to escape the predictions.
The conjecture remains unproven for the general case. The long version of this paper, however, mathematically justifies the conjecture for certain special cases and provides some intuitive justification for the general case (Schmidhuber, 1991). In addition, algorithms based solely on -maximization performed well in the experiments to be described below.