论文标题
$ \ MATHCAL {R} $的内在维度和概括属性 - 规范归纳偏差
Intrinsic dimensionality and generalization properties of the $\mathcal{R}$-norm inductive bias
论文作者
论文摘要
我们研究$ \ Mathcal {r} $的结构和统计属性 - 规范最小化由特定目标函数标记的数据集的插值。 $ \ Mathcal {r} $ - 标准是两层神经网络的感应偏差的基础，最近引入了捕获网络权重大小的功能效果，与网络宽度无关。我们发现，即使有适合数据的脊函数，这些插值也是本质上的多元函数，而且$ \ Mathcal {r} $ - 规范归纳偏见不足以实现某些学习问题的统计上最佳概括。总的来说，这些结果对与实用神经网络训练有关的感应偏差有了新的启示。
We study the structural and statistical properties of $\mathcal{R}$-norm minimizing interpolants of datasets labeled by specific target functions. The $\mathcal{R}$-norm is the basis of an inductive bias for two-layer neural networks, recently introduced to capture the functional effect of controlling the size of network weights, independently of the network width. We find that these interpolants are intrinsically multivariate functions, even when there are ridge functions that fit the data, and also that the $\mathcal{R}$-norm inductive bias is not sufficient for achieving statistically optimal generalization for certain learning problems. Altogether, these results shed new light on an inductive bias that is connected to practical neural network training.
