论文标题
双重复发神经网络中记忆和处理的分离
Separation of Memory and Processing in Dual Recurrent Neural Networks
论文作者
论文摘要
我们探索了一个神经网络体系结构，该架构堆叠了一个经常性层和一个也连接到输入的进料层，并将其与标准的Elman和LSTM体系结构在准确性和解释性方面进行比较。当将噪声引入复发单元的激活函数中时，这些神经元被迫进入二进制激活状态，使网络的行为与有限的自动机一样。最终的模型更简单，更容易解释和在不同的样本问题上获得更高的准确性，包括识别普通语言，在不同基础中的添加以及产生算术表达式的计算。
We explore a neural network architecture that stacks a recurrent layer and a feedforward layer that is also connected to the input, and compare it to standard Elman and LSTM architectures in terms of accuracy and interpretability. When noise is introduced into the activation function of the recurrent units, these neurons are forced into a binary activation regime that makes the networks behave much as finite automata. The resulting models are simpler, easier to interpret and get higher accuracy on different sample problems, including the recognition of regular languages, the computation of additions in different bases and the generation of arithmetic expressions.
