论文标题
直接暂时关系提取的鲁顿预训练的神经模型
Robustly Pre-trained Neural Model for Direct Temporal Relation Extraction
论文作者
论文摘要
背景：确定临床事件与时间表之间的关系是有意义地分析用于高级AI应用程序的临床文本的关键挑战。尽管存在先前的研究，但最先进的表现仍有很大的改进空间。 方法：我们研究了BERT的几种变体（使用变压器的双向编码器表示），其中一些涉及临床领域定制，而其他涉及改进的体系结构和/或培训策略。我们使用直接的时间关系数据集评估了这些方法，该数据集是2012 I2B2时间关系挑战数据集的语义集中子集。 结果：我们的结果表明，采用更好的培训策略（包括使用10倍较大的语料库）的罗伯塔（Roberta）将总体F量度提高了0.0864的绝对得分（以1.00比例为单位），因此相对于先前通过SVM（支持载体机）模型而实现的先前最新性能，相对于以前的最先进的性能，错误率将错误率提高了24％。 结论：现代上下文语言建模神经网络，在大型语料库中进行了预训练，即使在高度稳定的临床时间关系任务上也能够实现令人印象深刻的表现。
Background: Identifying relationships between clinical events and temporal expressions is a key challenge in meaningfully analyzing clinical text for use in advanced AI applications. While previous studies exist, the state-of-the-art performance has significant room for improvement. Methods: We studied several variants of BERT (Bidirectional Encoder Representations using Transformers) some involving clinical domain customization and the others involving improved architecture and/or training strategies. We evaluated these methods using a direct temporal relations dataset which is a semantically focused subset of the 2012 i2b2 temporal relations challenge dataset. Results: Our results show that RoBERTa, which employs better pre-training strategies including using 10x larger corpus, has improved overall F measure by 0.0864 absolute score (on the 1.00 scale) and thus reducing the error rate by 24% relative to the previous state-of-the-art performance achieved with an SVM (support vector machine) model. Conclusion: Modern contextual language modeling neural networks, pre-trained on a large corpus, achieve impressive performance even on highly-nuanced clinical temporal relation tasks.
