论文标题
过于拟合还是不合适？了解对抗训练的鲁棒性下降
Overfitting or Underfitting? Understand Robustness Drop in Adversarial Training
论文作者
论文摘要
我们的目标是了解为什么在进行对抗性训练太久后的鲁棒性会下降。尽管这种现象通常被解释为过度拟合，但我们的分析表明其主要原因是扰动不足。我们观察到，在训练太长时间后，FGSM生成的扰动会恶化为随机噪声。直觉上，由于没有进行参数更新来加强扰动发生器，因此一旦此过程崩溃，就可以将其捕获在这种本地Optima中。同样，复杂的这个过程可以避免稳健性下降，这支持这种现象是由于不足而不是过度拟合而引起的。鉴于我们的分析，我们提出了一个自适应的对抗训练框架，该培训框架可以参数扰动产生并逐步增强它们。屏蔽扰动不足，释放了我们框架的潜力。在我们的实验中，与PGD-10相比，分开提供了可比甚至更好的鲁棒性，其计算成本约为1/4。
Our goal is to understand why the robustness drops after conducting adversarial training for too long. Although this phenomenon is commonly explained as overfitting, our analysis suggest that its primary cause is perturbation underfitting. We observe that after training for too long, FGSM-generated perturbations deteriorate into random noise. Intuitively, since no parameter updates are made to strengthen the perturbation generator, once this process collapses, it could be trapped in such local optima. Also, sophisticating this process could mostly avoid the robustness drop, which supports that this phenomenon is caused by underfitting instead of overfitting. In the light of our analyses, we propose APART, an adaptive adversarial training framework, which parameterizes perturbation generation and progressively strengthens them. Shielding perturbations from underfitting unleashes the potential of our framework. In our experiments, APART provides comparable or even better robustness than PGD-10, with only about 1/4 of its computational cost.
