论文标题
浅网络的1条路线的有效近端映射
Efficient Proximal Mapping of the 1-path-norm of Shallow Networks
论文作者
论文摘要
我们展示了浅神经网络的1条路线的两个新的重要特性。首先，尽管具有非平滑度和非跨性别性，它允许可以有效地计算出封闭形式的近端运算符，从而允许使用随机近端梯度型方法用于正常的经验风险最小化。其次，当激活功能是可区分的时，它在网络的Lipschitz常数上提供了上限。这种界限比Lipschitz常数的微不足道层的产物更紧密，激励其用于训练网络可靠的对抗性扰动。在实际实验中，我们说明了使用近端映射的优势，并比较了Lipschitz常数（Parseval Networks）对1个路径，L1-norm和层的约束所引起的稳健性 - 准确性权衡。
We demonstrate two new important properties of the 1-path-norm of shallow neural networks. First, despite its non-smoothness and non-convexity it allows a closed form proximal operator which can be efficiently computed, allowing the use of stochastic proximal-gradient-type methods for regularized empirical risk minimization. Second, when the activation functions is differentiable, it provides an upper bound on the Lipschitz constant of the network. Such bound is tighter than the trivial layer-wise product of Lipschitz constants, motivating its use for training networks robust to adversarial perturbations. In practical experiments we illustrate the advantages of using the proximal mapping and we compare the robustness-accuracy trade-off induced by the 1-path-norm, L1-norm and layer-wise constraints on the Lipschitz constant (Parseval networks).
