For illustration purposes,
we assume that `knows' all possible action sequences
straight movements of the `animat', and
that the costs of all these action sequences
are already known by .
In that case it is easy to compute (1).
The start of the -th `sub-program' is
, its end point is
(1) becomes equal to the area
For the -th `sub-program',
is defined as
Consider figure 4. A single swamp has to be overcome by the `animat'. With 40 hidden nodes and a learning rate , a recurrent subgoal generator (architecture 2) needed 20 iterations to find a satisfactory solution.
Now consider figure 5. Multiple swamps separate the start from the goal. With 40 hidden nodes and a learning rate , a static subgoal generator (architecture 1) needed 22 iterations to find a satisfactory solution.