next up previous
Next: AN ERROR FUNCTION FOR Up: OBJECTIVE FUNCTIONS FOR THE Previous: AN ERROR FUNCTION FOR

AN OBJECTIVE FUNCTION FOR THE BINARY CRITERION

A well-known objective function $V$ for enforcing binary codes is given by

\begin{displaymath}
V =
\frac{1}{2}
\sum_i \sum_p (\bar{y_i} - y^p_i)^2.
\end{displaymath}

Maximizing this term encourages each unit to take on binary values. The contribution of each unit $i$ is maximized if $E(y_i)$ is as close to 0.5 as possible. This implies maximal entropy for unit $i$ under the binary constraint, i.e., $i$ wants to become a binary unit that conveys maximal information about its input.



Juergen Schmidhuber 2003-02-13


Back to Independent Component Analysis page.