论文标题
飞溅：可学习的激活功能，以提高准确性和对抗性鲁棒性
SPLASH: Learnable Activation Functions for Improving Accuracy and Adversarial Robustness
论文作者
论文摘要
我们介绍了Splash单元，这是一类可学习的激活功能，以同时提高深神经网络的准确性，同时也提高了它们对对抗性攻击的稳健性。飞溅单元既具有简单的参数化，并保持近似多种非线性函数的能力。飞溅单元是：1）连续； 2）接地（f（0）= 0）; 3）使用对称铰链； 4）铰链的位置直接源自数据（即无需学习）。与包括Relu及其变体在内的其他九个学习和固定的激活功能相比，Splash单元在三个数据集（MNIST，CIFAR-10和CIFAR-10和CIFAR-100）和四个架构（LENET5，All-CNN，Resnet-20和Network-In-In NetWork）中显示出卓越的性能。此外，我们表明，飞溅单位显着提高了深度神经网络对对抗性攻击的鲁棒性。我们在黑盒和开放式对抗攻击上进行的实验表明，通常使用Splash单位而不是Splash单位而不是Relus，通常使用LENET5，ALL-CNN，RESNET-20和网络网络的架构可以增强对抗性攻击的31％。
We introduce SPLASH units, a class of learnable activation functions shown to simultaneously improve the accuracy of deep neural networks while also improving their robustness to adversarial attacks. SPLASH units have both a simple parameterization and maintain the ability to approximate a wide range of non-linear functions. SPLASH units are: 1) continuous; 2) grounded (f(0) = 0); 3) use symmetric hinges; and 4) the locations of the hinges are derived directly from the data (i.e. no learning required). Compared to nine other learned and fixed activation functions, including ReLU and its variants, SPLASH units show superior performance across three datasets (MNIST, CIFAR-10, and CIFAR-100) and four architectures (LeNet5, All-CNN, ResNet-20, and Network-in-Network). Furthermore, we show that SPLASH units significantly increase the robustness of deep neural networks to adversarial attacks. Our experiments on both black-box and open-box adversarial attacks show that commonly-used architectures, namely LeNet5, All-CNN, ResNet-20, and Network-in-Network, can be up to 31% more robust to adversarial attacks by simply using SPLASH units instead of ReLUs.
