论文标题
Neuralef：深层神经网络解构内核
NeuralEF: Deconstructing Kernels by Deep Neural Networks
论文作者
论文摘要
学习由内核定义的积分运算符和数据分布的主要征函数是许多机器学习问题的核心。基于nyStr {Ö} m公式的传统非参数解决方案遭受可伸缩性问题。最近的工作已经采用了一种参数方法，即训练神经网络以近似征本征函数。但是，现有的方法依赖于昂贵的正交步骤，并且难以实施。我们表明，可以通过使用新的目标函数来解决这些问题，该目标函数概括了eigengame〜 \ citep {gemp2020 eigengame}以功能空间。我们在各种监督和无监督的学习问题上测试我们的方法，并表明它为多项式，径向基础，神经网络高斯过程和神经切线内核提供了准确的近似值。最后，我们证明我们的方法可以通过近似Gauss-Newton矩阵来扩展深神经网络对现代图像分类数据集的线性性拉普拉斯近似。代码可在\ url {https://github.com/thudzj/neuraleigenfunction}上找到。
Learning the principal eigenfunctions of an integral operator defined by a kernel and a data distribution is at the core of many machine learning problems. Traditional nonparametric solutions based on the Nystr{ö}m formula suffer from scalability issues. Recent work has resorted to a parametric approach, i.e., training neural networks to approximate the eigenfunctions. However, the existing method relies on an expensive orthogonalization step and is difficult to implement. We show that these problems can be fixed by using a new series of objective functions that generalizes the EigenGame~\citep{gemp2020eigengame} to function space. We test our method on a variety of supervised and unsupervised learning problems and show it provides accurate approximations to the eigenfunctions of polynomial, radial basis, neural network Gaussian process, and neural tangent kernels. Finally, we demonstrate our method can scale up linearised Laplace approximation of deep neural networks to modern image classification datasets through approximating the Gauss-Newton matrix. Code is available at \url{https://github.com/thudzj/neuraleigenfunction}.
