论文标题
纳顿额外：一种噪声自适应加速二阶方法的第一种方法
Extra-Newton: A First Approach to Noise-Adaptive Accelerated Second-Order Methods
论文作者
论文摘要
这项工作提出了一种通用和自适应的二阶方法，用于最大程度地减少二阶光滑，凸功能。当Oracle反馈随机带有差异$σ^2 $时，我们的算法就可以实现$ O（σ / \ sqrt {t}）$收敛，并将其收敛性提高到$ O（1 / t^3）$具有确定性的甲壳的$（1 / t^3）$，而$ t $是$ t $是迭代的数量。我们的方法还可以在不知道Oracle Apriori的性质的情况下插入这些速率，这是由无参数的自适应步骤尺寸启用的，该尺寸忽略了平滑度模量，方差界限和受约束集合的直径的知识。据我们所知，这是第一个具有二阶优化文献中这种全球保证的通用算法。
This work proposes a universal and adaptive second-order method for minimizing second-order smooth, convex functions. Our algorithm achieves $O(σ/ \sqrt{T})$ convergence when the oracle feedback is stochastic with variance $σ^2$, and improves its convergence to $O( 1 / T^3)$ with deterministic oracles, where $T$ is the number of iterations. Our method also interpolates these rates without knowing the nature of the oracle apriori, which is enabled by a parameter-free adaptive step-size that is oblivious to the knowledge of smoothness modulus, variance bounds and the diameter of the constrained set. To our knowledge, this is the first universal algorithm with such global guarantees within the second-order optimization literature.
