For illustration purposes,
we assume that `knows' all possible action sequences
leading to
*straight* movements of the `animat', and
that the costs of all these action sequences
are already known by .
In that case it is easy to compute (1).
The start of the -th `sub-program' is
, its end point is
.
(1) becomes equal to the area

(4) |

For the -th `sub-program',
is defined as

(5) |

Consider figure 4. A single swamp has to be overcome by the `animat'. With 40 hidden nodes and a learning rate , a recurrent subgoal generator (architecture 2) needed 20 iterations to find a satisfactory solution.

Now consider figure 5. Multiple swamps separate the start from the goal. With 40 hidden nodes and a learning rate , a static subgoal generator (architecture 1) needed 22 iterations to find a satisfactory solution.

Back to Subgoal learning - Hierarchical Learning

Pages with Subgoal learning pictures