论文标题
Riemannian随机梯度方法用于嵌套组成优化
Riemannian Stochastic Gradient Method for Nested Composition Optimization
论文作者
论文摘要
这项工作考虑了嵌套形式的功能组成优化，而每个函数都包含期望。这种类型的问题在于在元学习中的策略评估或元学习中的模型定制中越来越受欢迎。不能直接应用用于非复合优化的标准riemannian随机梯度方法，因为内部功能的随机近似在外部函数的梯度中造成了偏见。为了进行两级组合优化，我们提出了一种Riemannian随机组成梯度下降（R-SCGD）方法，该方法找到了近似固定点，预期的是平方的Riemannian梯度小于$ε$，in $ O（ε^{-2}）$呼叫$ o（ε^{ - 2}）$呼叫，呼叫具有外部功能和渐变功能和渐变功能和渐变功能和渐变功能和渐变功能和渐变功能和渐变功能和渐变功能或渐变功能和渐变功能和渐变功能和渐变功能和渐变作用为效果和渐变作用为所构成的作用和所致作用为所构成的作用为所构成的作用和渐变作用为所致而构成的。此外，我们将R-SCGD算法概括为多层嵌套组成结构的问题，对于一阶随机甲骨文而言，$ O（ε^{ - 2}）$的复杂性相同。最后，在强化学习中，对R-SCGD方法的性能进行了数值评估。
This work considers optimization of composition of functions in a nested form over Riemannian manifolds where each function contains an expectation. This type of problems is gaining popularity in applications such as policy evaluation in reinforcement learning or model customization in meta-learning. The standard Riemannian stochastic gradient methods for non-compositional optimization cannot be directly applied as stochastic approximation of inner functions create bias in the gradients of the outer functions. For two-level composition optimization, we present a Riemannian Stochastic Composition Gradient Descent (R-SCGD) method that finds an approximate stationary point, with expected squared Riemannian gradient smaller than $ε$, in $O(ε^{-2})$ calls to the stochastic gradient oracle of the outer function and stochastic function and gradient oracles of the inner function. Furthermore, we generalize the R-SCGD algorithms for problems with multi-level nested compositional structures, with the same complexity of $O(ε^{-2})$ for the first-order stochastic oracle. Finally, the performance of the R-SCGD method is numerically evaluated over a policy evaluation problem in reinforcement learning.
