论文标题
更多的数据可以扩大对抗性稳健模型和标准模型之间的概括差距
More Data Can Expand the Generalization Gap Between Adversarially Robust and Standard Models
论文作者
论文摘要
尽管在实践中取得了巨大的成功，但现代的机器学习模型很容易受到使人对数据的影响的对抗性攻击的影响，但导致严重且潜在的危险预测错误。为了解决这个问题，从业人员经常使用对抗性培训来学习对此类攻击的强大模型，而以不受干扰的测试集的较高概括错误为代价。传统的观点是，更多的培训数据应缩小对抗训练模型的概括误差和标准模型之间的差距。但是，我们研究了在$ \ ell_ \ infty $攻击下对高斯和伯努利模型的强大分类器的培训，并且我们证明，更多的数据实际上可能会增加此差距。此外，我们的理论结果确定了是否以及何时将最终开始缩小差距。最后，我们在实验上证明了我们的结果也适用于线性回归模型，这可能表明这种现象更广泛地发生。
Despite remarkable success in practice, modern machine learning models have been found to be susceptible to adversarial attacks that make human-imperceptible perturbations to the data, but result in serious and potentially dangerous prediction errors. To address this issue, practitioners often use adversarial training to learn models that are robust against such attacks at the cost of higher generalization error on unperturbed test sets. The conventional wisdom is that more training data should shrink the gap between the generalization error of adversarially-trained models and standard models. However, we study the training of robust classifiers for both Gaussian and Bernoulli models under $\ell_\infty$ attacks, and we prove that more data may actually increase this gap. Furthermore, our theoretical results identify if and when additional data will finally begin to shrink the gap. Lastly, we experimentally demonstrate that our results also hold for linear regression models, which may indicate that this phenomenon occurs more broadly.
