RCC Cannot Compute Certain FSA, Even with Arbitrary Transfer Functions, from Advances in Neural Information Processing Systems 10 (NIPS 10), 1997.

Existing proofs demonstrating the computational limitations of Recurrent Cascade Correlation and similar networks (Fahlman, 1991; Bachrach, 1988; Mozer, 1988) explicitly limit their results to units having sigmoidal or hard-threshold transfer functions (Giles et al., 1995; and Kremer, 1996).   The proof given here shows that for any finite, discrete transfer function used by the units of an RCC network, there are finite-state automata (FSA) that the network cannot model, no matter how many units are used. The proof also applies to continuous transfer functions with a finite number of fixed-points, such as sigmoid and radial-basis functions.