论文标题
将深Q网络转换为事件驱动的尖峰神经网络的策略和基准
Strategy and Benchmark for Converting Deep Q-Networks to Event-Driven Spiking Neural Networks
论文作者
论文摘要
尖峰神经网络（SNN）具有在专用的神经形态硬件上对深度神经网络（DNN）实施节能的巨大潜力。最近的研究表明，与图像分类任务（包括CIFAR-10和Imagenet数据）相比，SNN与DNN相比的竞争性能。目前的工作重点是将SNN与Atari游戏中的深入增强学习结合使用，与图像分类相比，该游戏涉及额外的复杂性。我们回顾了将DNN转换为SNN并将转换扩展到深Q-Networks（DQN）的理论。我们建议对发射速率的强大表示，以减少转换过程中的误差。此外，我们引入了一个新的指标，以分别比较DQN和SNN做出的决策来评估转换过程。我们还分析了模拟时间和参数归一化如何影响转换后的SNN的性能。我们在17场表现最佳的Atari比赛中取得了竞争成绩。据我们所知，我们的工作是第一个在使用SNN的多个Atari游戏中实现最先进的表现的工作。我们的工作是将DQN转换为SNN的基准，并为通过SNNS解决强化学习任务的进一步研究铺平了道路。
Spiking neural networks (SNNs) have great potential for energy-efficient implementation of Deep Neural Networks (DNNs) on dedicated neuromorphic hardware. Recent studies demonstrated competitive performance of SNNs compared with DNNs on image classification tasks, including CIFAR-10 and ImageNet data. The present work focuses on using SNNs in combination with deep reinforcement learning in ATARI games, which involves additional complexity as compared to image classification. We review the theory of converting DNNs to SNNs and extending the conversion to Deep Q-Networks (DQNs). We propose a robust representation of the firing rate to reduce the error during the conversion process. In addition, we introduce a new metric to evaluate the conversion process by comparing the decisions made by the DQN and SNN, respectively. We also analyze how the simulation time and parameter normalization influence the performance of converted SNNs. We achieve competitive scores on 17 top-performing Atari games. To the best of our knowledge, our work is the first to achieve state-of-the-art performance on multiple Atari games with SNNs. Our work serves as a benchmark for the conversion of DQNs to SNNs and paves the way for further research on solving reinforcement learning tasks with SNNs.
