论文标题
张量火车分解在复发网络上
Tensor train decompositions on recurrent networks
论文作者
论文摘要
在众多日常实时任务（例如语音，语言，视频和多模式学习）中，诸如长期记忆（LSTM）网络之类的经常性神经网络（RNN）至关重要。从云到边缘计算的转移加剧了包含RNN参数增长的需求。当前对RNN的研究表明，尽管在卷积神经网络（CNN）上获得了表现，但在压缩RNN中保持良好的性能仍然是一个挑战。有关压缩的大多数文献都侧重于使用矩阵产品（MPO）操作员张量列车上的CNN。但是，就储存量减少和计算时间而言，矩阵产品状态（MPS）张量火车比MPO具有更具吸引力的功能。我们表明，MPS张量列车应通过NLP任务的理论分析和实际实验在LSTM网络压缩的最前沿。
Recurrent neural networks (RNN) such as long-short-term memory (LSTM) networks are essential in a multitude of daily live tasks such as speech, language, video, and multimodal learning. The shift from cloud to edge computation intensifies the need to contain the growth of RNN parameters. Current research on RNN shows that despite the performance obtained on convolutional neural networks (CNN), keeping a good performance in compressed RNNs is still a challenge. Most of the literature on compression focuses on CNNs using matrix product (MPO) operator tensor trains. However, matrix product state (MPS) tensor trains have more attractive features than MPOs, in terms of storage reduction and computing time at inference. We show that MPS tensor trains should be at the forefront of LSTM network compression through a theoretical analysis and practical experiments on NLP task.
