论文标题
深层目标推理的分层增强学习：表达分析
Hierarchical Reinforcement Learning for Deep Goal Reasoning: An Expressiveness Analysis
论文作者
论文摘要
层次DQN（H-DQN）是前馈神经网络的两级体系结构，其中元级别选择目标，较低级别采取行动来实现目标。我们显示了无法通过H-DQN解决的任务，这体现了这种类型的分层框架（HF）的限制。我们描述了经常性的分层框架（RHF），概括了在元级别使用经常性神经网络的体系结构。我们使用上下文敏感语法分析了HF和RHF的表现力。我们表明RHF比HF更具表现力。我们执行将RHF与两个HF基准的实现进行比较的实验。结果证实了我们的理论发现。
Hierarchical DQN (h-DQN) is a two-level architecture of feedforward neural networks where the meta level selects goals and the lower level takes actions to achieve the goals. We show tasks that cannot be solved by h-DQN, exemplifying the limitation of this type of hierarchical framework (HF). We describe the recurrent hierarchical framework (RHF), generalizing architectures that use a recurrent neural network at the meta level. We analyze the expressiveness of HF and RHF using context-sensitive grammars. We show that RHF is more expressive than HF. We perform experiments comparing an implementation of RHF with two HF baselines; the results corroborate our theoretical findings.
