论文标题
学会用内存压缩和转移总结长文本
Learning to Summarize Long Texts with Memory Compression and Transfer
论文作者
论文摘要
我们介绍了MEM2MEM，这是一种用于层次复发神经网络基于编码器解码器架构的内存到内存机制，我们探索了其用于抽象文档摘要的用途。 MEM2MEM通过可读/可读/可写的外部内存模块转移“记忆”，从而增加编码器和解码器。我们的内存正则化将编码的输入文章压缩为更紧凑的句子表示。最重要的是，内存压缩步骤执行隐式提取，而无需标记，避开了次优地面真相数据的问题，以及混合抽取吸取性摘要技术的暴露偏置。通过允许解码器在编码的输入内存上读取/写入，该模型学会了读取有关输入文章的显着信息，同时跟踪已生成的内容。我们的MEM2MEM方法产生的结果与基于最新变压器的摘要方法具有竞争力，但参数少16倍
We introduce Mem2Mem, a memory-to-memory mechanism for hierarchical recurrent neural network based encoder decoder architectures and we explore its use for abstractive document summarization. Mem2Mem transfers "memories" via readable/writable external memory modules that augment both the encoder and decoder. Our memory regularization compresses an encoded input article into a more compact set of sentence representations. Most importantly, the memory compression step performs implicit extraction without labels, sidestepping issues with suboptimal ground-truth data and exposure bias of hybrid extractive-abstractive summarization techniques. By allowing the decoder to read/write over the encoded input memory, the model learns to read salient information about the input article while keeping track of what has been generated. Our Mem2Mem approach yields results that are competitive with state of the art transformer based summarization methods, but with 16 times fewer parameters
