论文标题
预先形式：长期时间序列的多尺度段相关性的预测变压器预测
Preformer: Predictive Transformer with Multi-Scale Segment-wise Correlations for Long-Term Time Series Forecasting
论文作者
论文摘要
基于变压器的方法在长期时间序列预测中显示出巨大的潜力。但是，这些方法中的大多数都采用了标准的自我注意力发项机制，这不仅在长期预测方面变得很棘手，因为它的复杂性随时间序列的长度而倍增，而且无法从上下文中明确捕获预测依赖性，因为相应的键和值从同一点转化为相应的键和值。本文提出了一个基于预测变压器的模型，称为{\ em preformer}。 Preformer引入了一种新颖的有效{\ em多尺度段 - 相关}机制，该机制将时间序列分为段，并利用基于细分的基于段的相关性注意编码时间序列。开发了多尺度结构，以在不同的时间尺度上汇总依赖关系，并促进段长度的选择。 Preformer进一步设计了用于解码的预测范式，其中键和值来自两个连续的段，而不是同一段。这样，如果密钥段与查询段具有很高的相关得分，则其连续段对查询段的预测有更多的贡献。广泛的实验表明，我们的预构物表现优于其他基于变压器的方法。
Transformer-based methods have shown great potential in long-term time series forecasting. However, most of these methods adopt the standard point-wise self-attention mechanism, which not only becomes intractable for long-term forecasting since its complexity increases quadratically with the length of time series, but also cannot explicitly capture the predictive dependencies from contexts since the corresponding key and value are transformed from the same point. This paper proposes a predictive Transformer-based model called {\em Preformer}. Preformer introduces a novel efficient {\em Multi-Scale Segment-Correlation} mechanism that divides time series into segments and utilizes segment-wise correlation-based attention for encoding time series. A multi-scale structure is developed to aggregate dependencies at different temporal scales and facilitate the selection of segment length. Preformer further designs a predictive paradigm for decoding, where the key and value come from two successive segments rather than the same segment. In this way, if a key segment has a high correlation score with the query segment, its successive segment contributes more to the prediction of the query segment. Extensive experiments demonstrate that our Preformer outperforms other Transformer-based methods.
