论文标题
神经网络学习对称功能的功能视角
A Functional Perspective on Learning Symmetric Functions with Neural Networks
论文作者
论文摘要
对称函数将其视为无序的，固定尺寸的集合，已知可以通过执行置换不变性的神经网络普遍表示。这些体系结构仅提供固定输入大小的保证，但是在许多实际应用中，包括点云和粒子物理，相关的概括概念应包括改变输入大小。在这项工作中，我们将对称函数（任何大小）视为概率度量的功能，并研究了定义在措施上定义的神经网络的学习和表示。通过关注浅层架构，我们在不同选择的正则化选择（例如RKH和变异规范）下建立了近似和概括界限，该范围捕获了功能空间的层次结构，而非线性学习程度越来越高。可以有效地学习产生的模型，并享受概括性的保证，可以按照经验验证，可以扩展跨输入尺寸。
Symmetric functions, which take as input an unordered, fixed-size set, are known to be universally representable by neural networks that enforce permutation invariance. These architectures only give guarantees for fixed input sizes, yet in many practical applications, including point clouds and particle physics, a relevant notion of generalization should include varying the input size. In this work we treat symmetric functions (of any size) as functions over probability measures, and study the learning and representation of neural networks defined on measures. By focusing on shallow architectures, we establish approximation and generalization bounds under different choices of regularization (such as RKHS and variation norms), that capture a hierarchy of functional spaces with increasing degree of non-linear learning. The resulting models can be learned efficiently and enjoy generalization guarantees that extend across input sizes, as we verify empirically.
