Certain FSA, Even with Arbitrary Transfer Functions, from
in Neural Information Processing Systems 10 (NIPS 10), 1997.
Existing proofs demonstrating the computational limitations of
Recurrent Cascade Correlation and similar networks (Fahlman, 1991;
Bachrach, 1988; Mozer, 1988) explicitly limit their results to units
having sigmoidal or hard-threshold transfer functions (Giles et al.,
1995; and Kremer, 1996). The proof given here shows that for any finite, discrete transfer
function used by the units of an RCC network, there are finite-state
automata (FSA) that the network cannot model, no matter how many units
are used. The proof also applies to continuous
transfer functions with a finite number of fixed-points, such as
sigmoid and radial-basis functions.