** Next:** SPECIAL CASE: LINEAR OUTPUT
** Up:** EFFECTS OF THE ADDITIONAL
** Previous:** First term of favors

##

Second term favors few, separated, common component
functions.

The term

punishes units with similar influence on the output.
We reformulate it:

Using

this can be rewritten as

For

holds. We obtain

We observe:
(1) an output unit that is very sensitive with respect
to two given hidden units
will heavily contribute to (compare
the numerator in the last term in the brackets of ).
(2) This large contribution can be reduced
by making both hidden units have large impact on other output units
(see denominator in the last term in the brackets of ).
**Choice of component functions (CFs).**
FMS tries to figure out a way of using
(1) as few CFs
as possible for each output unit (this
leads to separation of CFs),
while simultaneously (2) using the same CFs for as many output
units as possible (common CFs).

**Subsections**

** Next:** SPECIAL CASE: LINEAR OUTPUT
** Up:** EFFECTS OF THE ADDITIONAL
** Previous:** First term of favors
Juergen Schmidhuber
2003-02-13

Back to Independent Component Analysis page.