论文标题
通过Riemannian收缩在监督学习中的概括
Generalization in Supervised Learning Through Riemannian Contraction
论文作者
论文摘要
我们证明，在监督学习环境中的Riemannian收缩意味着概括。具体来说，我们表明，如果优化器在某些带有速率$λ> 0 $的Riemannian度量中收缩，则它在算法上均匀地稳定，并具有$ \ Mathcal {o}（1/λn）$，其中$ n $是训练集中标记的示例的数量。在连续和离散时间的随机和确定性优化的结果，对于凸和非凸损失表面。在特定的梯度下降或强烈凸出损失表面的特定情况下，相关的概括范围减少了众所周知的结果。它们可以在某些线性设置中表现出最佳状态，例如梯度流下的内核脊回归。
We prove that Riemannian contraction in a supervised learning setting implies generalization. Specifically, we show that if an optimizer is contracting in some Riemannian metric with rate $λ> 0$, it is uniformly algorithmically stable with rate $\mathcal{O}(1/λn)$, where $n$ is the number of labelled examples in the training set. The results hold for stochastic and deterministic optimization, in both continuous and discrete-time, for convex and non-convex loss surfaces. The associated generalization bounds reduce to well-known results in the particular case of gradient descent over convex or strongly convex loss surfaces. They can be shown to be optimal in certain linear settings, such as kernel ridge regression under gradient flow.
