论文标题
平行层次变压器，具有抽象性多文件摘要的注意对齐
Parallel Hierarchical Transformer with Attention Alignment for Abstractive Multi-Document Summarization
论文作者
论文摘要
与单案摘要相比，抽象性多文章摘要（MDS）在其冗长和链接的来源的表示和覆盖范围内提出了挑战。这项研究开发了一个平行的分层变压器（PHT），并具有MDS的注意对齐。通过结合单词和段落级的多头注意，PHT的层次结构可以更好地处理令牌和文档级别的依赖项。为了指导解码到更好的源文档覆盖范围，然后将注意力调整机制引入以校准光束搜索，并预测的最佳注意力分布。根据Wikisum数据，进行了全面的评估，以测试拟议的体系结构对MD的改进。通过更好地处理内部和跨文档的信息，胭脂和人类评估的结果都表明，我们的分层模型以相对较低的计算成本生成较高质量的摘要。
In comparison to single-document summarization, abstractive Multi-Document Summarization (MDS) brings challenges on the representation and coverage of its lengthy and linked sources. This study develops a Parallel Hierarchical Transformer (PHT) with attention alignment for MDS. By incorporating word- and paragraph-level multi-head attentions, the hierarchical architecture of PHT allows better processing of dependencies at both token and document levels. To guide the decoding towards a better coverage of the source documents, the attention-alignment mechanism is then introduced to calibrate beam search with predicted optimal attention distributions. Based on the WikiSum data, a comprehensive evaluation is conducted to test improvements on MDS by the proposed architecture. By better handling the inner- and cross-document information, results in both ROUGE and human evaluation suggest that our hierarchical model generates summaries of higher quality relative to other Transformer-based baselines at relatively low computational cost.
