is the -th unit in the network. is the -th non-input unit in the network. is the -th `normal' input unit in the network. is the -th `normal' output unit. If stands for a unit, then is its differentiable activation function and 's activation at time is denoted by . If stands for a vector, then is the -th component of .

Each input unit has a directed connection to each non-input unit. Each non-input unit has a directed connection to each non-input unit. There are connections in the network. The connection from unit to unit is denoted by . For instance, one of the names of the connection from the -th `normal' input unit to the the -th `normal' output unit is . 's real-valued weight at time is denoted by . Before training, all weights are randomly initialized.

The following features are needed to obtain `self-reference'.
*Details of the network dynamics follow in the next section.*

1. The network receives performance information through the
*eval units*, which are special input units.
is the -th eval unit (of such units)
in the network.

2. Each connection of the net gets an address.
One way of doing this is to
introduce a *binary* address, , for
each connection . This will help the network to
do computations concerning
its own *weights* in terms of *activations*, as
will be seen later.

3. is the -th *analyzing unit* (of
such units,
where returns the first integer ).
The analyzing units are special non-input units.
They serve to indicate which connections the current
algorithm of the network (defined by the current weight matrix plus
the current activations) will access next (see next section).
A special input unit for reading current weight values
that is used in conjunction with the analyzing units
is called .

4. The network may modify any of its weights.
Some non-input units that are not `normal' output
units or analyzing units are called the *modifying units*.
is the -th modifying unit (of
such units).
The modifying units serve to
address connections to be modified.
A special output unit for modifying weights (used in conjunction
with the modifying units, see next section)
is called
.
should allow both positive
and negative activations
.

Back to Metalearning page

Back to Recurrent Neural Networks page