论文标题
关于卷积神经切线和高斯过程内核的光谱偏差
On the Spectral Bias of Convolutional Neural Tangent and Gaussian Process Kernels
论文作者
论文摘要
我们通过各自的高斯过程和神经切线内核来研究各种过度参数化的卷积神经体系结构的特性。我们证明，随着归一化的多通道输入和relu激活，这些内核的特征函数由均匀度量的量形成，由球形谐波的产物形成，在不同像素的通道上定义。接下来，我们使用层次化的可分解内核来绑定其各自的特征值。我们表明，特征值衰减在多个一级，量化衰减速率，并得出反映这些网络中层次特征组成的度量。我们的结果提供了过度参数化卷积网络体系结构的具体定量表征。
We study the properties of various over-parametrized convolutional neural architectures through their respective Gaussian process and neural tangent kernels. We prove that, with normalized multi-channel input and ReLU activation, the eigenfunctions of these kernels with the uniform measure are formed by products of spherical harmonics, defined over the channels of the different pixels. We next use hierarchical factorizable kernels to bound their respective eigenvalues. We show that the eigenvalues decay polynomially, quantify the rate of decay, and derive measures that reflect the composition of hierarchical features in these networks. Our results provide concrete quantitative characterization of over-parameterized convolutional network architectures.
