论文标题
MS-RNN：时空预测学习的灵活多尺度框架
MS-RNN: A Flexible Multi-Scale Framework for Spatiotemporal Predictive Learning
论文作者
论文摘要
时空预测学习通过深度学习的帮助来预测未来的框架，在许多领域都广泛使用。先前的工作实质上是通过扩大或加深网络来改善模型性能，但它也带来了飙升的内存开销，这严重阻碍了该技术的开发和应用。为了提高性能而不增加内存消耗，我们专注于规模，这是提高模型性能但记忆要求低的另一个维度。在许多基于CNN的任务（例如图像分类和语义分割）中广泛证明了该有效性，但是在最近的RNN模型中尚未完全探索它。在本文中，我们从多尺度的好处中学习，我们提出了一个名为多尺度RNN（MS-RNN）的通用框架，以增强时空预测性学习的最新RNN模型。 We verify the MS-RNN framework by thorough theoretical analyses and exhaustive experiments, where the theory focuses on memory reduction and performance improvement while the experiments employ eight RNN models (ConvLSTM, TrajGRU, PredRNN, PredRNN++, MIM, MotionRNN, PredRNN-V2, and PrecipLSTM) and four datasets (Moving MNIST, TaxiBJ, KTH, and Germany).结果表明，RNN模型合并我们的框架的效率要低得多，但性能比以前更好。我们的代码在\ url {https://github.com/mazhf/ms-rnn}上发布。
Spatiotemporal predictive learning, which predicts future frames through historical prior knowledge with the aid of deep learning, is widely used in many fields. Previous work essentially improves the model performance by widening or deepening the network, but it also brings surging memory overhead, which seriously hinders the development and application of this technology. In order to improve the performance without increasing memory consumption, we focus on scale, which is another dimension to improve model performance but with low memory requirement. The effectiveness has been widely demonstrated in many CNN-based tasks such as image classification and semantic segmentation, but it has not been fully explored in recent RNN models. In this paper, learning from the benefit of multi-scale, we propose a general framework named Multi-Scale RNN (MS-RNN) to boost recent RNN models for spatiotemporal predictive learning. We verify the MS-RNN framework by thorough theoretical analyses and exhaustive experiments, where the theory focuses on memory reduction and performance improvement while the experiments employ eight RNN models (ConvLSTM, TrajGRU, PredRNN, PredRNN++, MIM, MotionRNN, PredRNN-V2, and PrecipLSTM) and four datasets (Moving MNIST, TaxiBJ, KTH, and Germany). The results show the efficiency that RNN models incorporating our framework have much lower memory cost but better performance than before. Our code is released at \url{https://github.com/mazhf/MS-RNN}.
