论文标题
使用子线性参数通过深层神经网络可证明的记忆
Provable Memorization via Deep Neural Networks using Sub-linear Parameters
论文作者
论文摘要
众所周知，$ o（n）$参数足以让神经网络记住任意$ n $输入标签对。通过利用深度，我们表明$ o（n^{2/3}）$参数足以记住$ n $ pairs，在温和的条件下，在输入点的分离下。特别是，显示更深的网络（即使宽度$ 3 $）也显示出比浅网络比浅网络更多的对，这也与最新的作品有关的作品一致，该作品对功能近似的深度效益。我们还提供了支持我们理论发现的经验结果。
It is known that $O(N)$ parameters are sufficient for neural networks to memorize arbitrary $N$ input-label pairs. By exploiting depth, we show that $O(N^{2/3})$ parameters suffice to memorize $N$ pairs, under a mild condition on the separation of input points. In particular, deeper networks (even with width $3$) are shown to memorize more pairs than shallow networks, which also agrees with the recent line of works on the benefits of depth for function approximation. We also provide empirical results that support our theoretical findings.
