论文标题
基于动力学系统的神经网络
Dynamical systems' based neural networks
论文作者
论文摘要
神经网络因其在许多应用中的有效性而引起了人们的兴趣。但是，它们的数学特性通常不太了解。如果数据固有的一些基本几何结构或近似函数的函数，则通常需要在神经网络的设计中考虑到这一点。在这项工作中，我们从非自主颂歌开始，并使用合适的，结构的，数值的时间差异来构建神经网络。然后从ode矢量场的性质推断神经网络的结构。除了将更多的结构注入网络体系结构外，此建模过程还可以更好地理解其行为。我们提出了两个通用近似结果，并演示了如何在神经网络上施加一些特定特性。特别的重点是1-卢比奇架构，包括不是1- lipschitz的层。如CIFAR-10和CIFAR-100数据集所示，这些网络在对抗攻击方面具有表现力和鲁棒性。
Neural networks have gained much interest because of their effectiveness in many applications. However, their mathematical properties are generally not well understood. If there is some underlying geometric structure inherent to the data or to the function to approximate, it is often desirable to take this into account in the design of the neural network. In this work, we start with a non-autonomous ODE and build neural networks using a suitable, structure-preserving, numerical time-discretisation. The structure of the neural network is then inferred from the properties of the ODE vector field. Besides injecting more structure into the network architectures, this modelling procedure allows a better theoretical understanding of their behaviour. We present two universal approximation results and demonstrate how to impose some particular properties on the neural networks. A particular focus is on 1-Lipschitz architectures including layers that are not 1-Lipschitz. These networks are expressive and robust against adversarial attacks, as shown for the CIFAR-10 and CIFAR-100 datasets.
