论文标题
深度强化学习中稀疏培训的状态
The State of Sparse Training in Deep Reinforcement Learning
论文作者
论文摘要
近年来，稀疏神经网络的使用迅速增长，尤其是在计算机视觉中。它们的吸引力在很大程度上源于训练和存储所需的参数数量以及学习效率的提高。有些令人惊讶的是，很少有努力探索他们在深度强化学习中的使用（DRL）。在这项工作中，我们进行了系统的调查，以应用多种现有的稀疏训练技术对各种DRL代理和环境。我们的结果证实了计算机视觉域中稀疏训练的发现 - 稀疏网络在DRL域中对相同的参数计数的稀疏网络表现更好。我们提供了有关DRL中各种组成部分如何受到稀疏网络的影响的详细分析，并通过建议有望提高稀疏训练方法的有效性以及推进其在DRL中的使用来结论。
The use of sparse neural networks has seen rapid growth in recent years, particularly in computer vision. Their appeal stems largely from the reduced number of parameters required to train and store, as well as in an increase in learning efficiency. Somewhat surprisingly, there have been very few efforts exploring their use in Deep Reinforcement Learning (DRL). In this work we perform a systematic investigation into applying a number of existing sparse training techniques on a variety of DRL agents and environments. Our results corroborate the findings from sparse training in the computer vision domain - sparse networks perform better than dense networks for the same parameter count - in the DRL domain. We provide detailed analyses on how the various components in DRL are affected by the use of sparse networks and conclude by suggesting promising avenues for improving the effectiveness of sparse training methods, as well as for advancing their use in DRL.
