is the -th unit in the network. is the -th non-input unit in the network. is the -th `normal' input unit in the network. is the -th `normal' output unit. If stands for a unit, then is its differentiable activation function and 's activation at time is denoted by . If stands for a vector, then is the -th component of .
Each input unit has a directed connection to each non-input unit. Each non-input unit has a directed connection to each non-input unit. There are connections in the network. The connection from unit to unit is denoted by . For instance, one of the names of the connection from the -th `normal' input unit to the the -th `normal' output unit is . 's real-valued weight at time is denoted by . Before training, all weights are randomly initialized.
The following features are needed to obtain `self-reference'. Details of the network dynamics follow in the next section.
1. The network receives performance information through the eval units, which are special input units. is the -th eval unit (of such units) in the network.
2. Each connection of the net gets an address. One way of doing this is to introduce a binary address, , for each connection . This will help the network to do computations concerning its own weights in terms of activations, as will be seen later.
3. is the -th analyzing unit (of such units, where returns the first integer ). The analyzing units are special non-input units. They serve to indicate which connections the current algorithm of the network (defined by the current weight matrix plus the current activations) will access next (see next section). A special input unit for reading current weight values that is used in conjunction with the analyzing units is called .
4. The network may modify any of its weights. Some non-input units that are not `normal' output units or analyzing units are called the modifying units. is the -th modifying unit (of such units). The modifying units serve to address connections to be modified. A special output unit for modifying weights (used in conjunction with the modifying units, see next section) is called . should allow both positive and negative activations .