LIMITATIONS

Generalization performance. In most non-trivial cases, the approach did not generalize very well. After training $S$ on a range of different subgoal generation tasks (various randomly generated start/goal combinations), the subgoals emitted in response to previously unseen problems often were far from being optimal. More research needs to be directed towards improving generalization performance.

Another limitation of our approach has been mentioned above: It relies on differentiable (although possibly adaptive) models of the costs associated with known action sequences. The domain knowledge resides in these models - from there it is extracted by the subgoal generation process. There are domains, however, where a differentiable evaluator module might be inappropriate or difficult to obtain.

Even in cases where there is a differentiable model at hand the problem of local minima remains. Local minima did not play a major role with the simple experiments described above - with large scale applications, however, some way of dealing with suboptimal solutions needs to be introduced.

Back to Subgoal learning - Hierarchical Learning
Pages with Subgoal learning pictures