论文标题
对称神经网络中的指数分离
Exponential Separations in Symmetric Neural Networks
论文作者
论文摘要
在这项工作中，我们展示了对称神经网络体系结构之间的新型分离。具体而言，我们将关系网络〜\ parencite {santoro2017simple}架构视为对深度群体的自然概括〜\ parencite {zaheer2017deep}架构，并研究了他们的表示差距。在限制分析激活函数的限制下，我们构建了一个对称函数，该功能在尺寸$ n $的集合上具有尺寸$ d $中的元素，以前的架构可以有效地近似，但事实证明，后者需要$ n $和$ d $的宽度指数。
In this work we demonstrate a novel separation between symmetric neural network architectures. Specifically, we consider the Relational Network~\parencite{santoro2017simple} architecture as a natural generalization of the DeepSets~\parencite{zaheer2017deep} architecture, and study their representational gap. Under the restriction to analytic activation functions, we construct a symmetric function acting on sets of size $N$ with elements in dimension $D$, which can be efficiently approximated by the former architecture, but provably requires width exponential in $N$ and $D$ for the latter.
