论文标题
对木马的注意力异常的研究
A Study of the Attention Abnormality in Trojaned BERTs
论文作者
论文摘要
特洛伊木马攻击引起了严重的安全问题。在本文中，我们研究了Trojaned Bert模型的潜在机制。我们观察到木马模型的注意力焦点漂移行为，即，在遇到中毒输入时，触发令牌劫持了注意力的焦点，无论上下文如何。我们对这种现象提供了彻底的定性和定量分析，从而揭示了对特洛伊木马机制的见解。基于观察结果，我们提出了一个基于注意力的特洛伊木马检测器，以将木马模型与干净的模型区分开。据我们所知，这是第一篇分析特洛伊木马机制并根据变压器注意的特洛伊木马检测器的论文。
Trojan attacks raise serious security concerns. In this paper, we investigate the underlying mechanism of Trojaned BERT models. We observe the attention focus drifting behavior of Trojaned models, i.e., when encountering an poisoned input, the trigger token hijacks the attention focus regardless of the context. We provide a thorough qualitative and quantitative analysis of this phenomenon, revealing insights into the Trojan mechanism. Based on the observation, we propose an attention-based Trojan detector to distinguish Trojaned models from clean ones. To the best of our knowledge, this is the first paper to analyze the Trojan mechanism and to develop a Trojan detector based on the transformer's attention.
