论文标题
DHRL：一种基于图形的长马和稀疏分层增强学习的方法
DHRL: A Graph-Based Approach for Long-Horizon and Sparse Hierarchical Reinforcement Learning
论文作者
论文摘要
分层增强学习（HRL）通过利用时间抽象在复杂的控制任务中取得了显着进步。但是，随着环境变大，以前的HRL算法通常会遭受严重的数据效率低下。扩展的组件，即$，目标空间和情节长度，对一个或高级和低级政策施加负担，因为这两个级别都共享了情节的总范围。在本文中，我们提出了一种使用层次增强学习中图（DHRL）的图形解耦的方法，该方法可以通过将高级和低级策略的地平线解耦来减轻此问题，并使用图形使用图形之间弥合两个视野的长度之间的间隙。 DHRL提供了可自由拉伸的高级动作间隔，在复杂的任务中促进了更长的时间抽象和更快的训练。在典型的HRL环境中，我们的方法优于最先进的HRL算法。此外，DHRL实现了漫长而复杂的运动和操纵任务。
Hierarchical Reinforcement Learning (HRL) has made notable progress in complex control tasks by leveraging temporal abstraction. However, previous HRL algorithms often suffer from serious data inefficiency as environments get large. The extended components, $i.e.$, goal space and length of episodes, impose a burden on either one or both high-level and low-level policies since both levels share the total horizon of the episode. In this paper, we present a method of Decoupling Horizons Using a Graph in Hierarchical Reinforcement Learning (DHRL) which can alleviate this problem by decoupling the horizons of high-level and low-level policies and bridging the gap between the length of both horizons using a graph. DHRL provides a freely stretchable high-level action interval, which facilitates longer temporal abstraction and faster training in complex tasks. Our method outperforms state-of-the-art HRL algorithms in typical HRL environments. Moreover, DHRL achieves long and complex locomotion and manipulation tasks.
