论文标题
一种基于时间和空间的本地峰值反向传播算法，可以在硬件中进行培训
A temporally and spatially local spike-based backpropagation algorithm to enable training in hardware
论文作者
论文摘要
尖峰神经网络（SNN）已成为用于分类任务的硬件有效体系结构。基于尖峰的编码的挑战是缺乏完全使用尖峰执行的通用训练机制。已经进行了几项尝试，用于采用在非加速人工神经网络（ANN）中使用的强大反向传播（BP）技术：（1）SNN可以通过外部计算的数值梯度来训练。 （2）基于天然尖峰的学习的主要进步是使用使用尖峰时代依赖性可塑性（STDP）使用分阶段向前/向后传递的近似反向传播。但是，在此类阶段之间进行梯度和重量更新计算的信息传递需要外部记忆和计算访问。这是标准神经形态硬件实现的挑战。在本文中，我们提出了一种基于随机SNN的后prop（SSNN-BP）算法，该算法利用复合神经元同时计算带有尖峰的前向通行激活和向后传递梯度。尽管签名的梯度值是基于SPIKE的表示的挑战，但我们通过将梯度信号分为正和负面流来解决这一问题。我们表明，我们的方法使用足够长的尖峰训练来接近BP ANN基线。最后，我们表明，可以通过强制执行获胜者的抑制性横向连接来实现良好表现的软磁体跨膜丢失函数。我们带有2层网络的SNN通过与MNIST，Fashion-MNIST，扩展MNIST和时间上编码的图像数据集（如NeuroMormorphic Mnist Dataset）等静态图像数据集上具有等效架构和正则参数的ANN相当地表现出出色的概括。因此，SSNN-BP可以使BP与纯粹基于尖峰的神经形态硬件兼容。
Spiking Neural Networks (SNNs) have emerged as a hardware efficient architecture for classification tasks. The challenge of spike-based encoding has been the lack of a universal training mechanism performed entirely using spikes. There have been several attempts to adopt the powerful backpropagation (BP) technique used in non-spiking artificial neural networks (ANN): (1) SNNs can be trained by externally computed numerical gradients. (2) A major advancement towards native spike-based learning has been the use of approximate Backpropagation using spike-time dependent plasticity (STDP) with phased forward/backward passes. However, the transfer of information between such phases for gradient and weight update calculation necessitates external memory and computational access. This is a challenge for standard neuromorphic hardware implementations. In this paper, we propose a stochastic SNN based Back-Prop (SSNN-BP) algorithm that utilizes a composite neuron to simultaneously compute the forward pass activations and backward pass gradients explicitly with spikes. Although signed gradient values are a challenge for spike-based representation, we tackle this by splitting the gradient signal into positive and negative streams. We show that our method approaches BP ANN baseline with sufficiently long spike-trains. Finally, we show that the well-performing softmax cross-entropy loss function can be implemented through inhibitory lateral connections enforcing a Winner Take All (WTA) rule. Our SNN with a 2-layer network shows excellent generalization through comparable performance to ANNs with equivalent architecture and regularization parameters on static image datasets like MNIST, Fashion-MNIST, Extended MNIST, and temporally encoded image datasets like Neuromorphic MNIST datasets. Thus, SSNN-BP enables BP compatible with purely spike-based neuromorphic hardware.
