论文标题
用单步内部方法线性收敛的双层优化
Linearly convergent bilevel optimization with single-step inner methods
论文作者
论文摘要
我们提出了一种解决双重优化问题的新方法，并以牛顿型方法求解全系统最佳条件，并将内部问题视为隐式功能。总体想法是解决全系统最佳条件，但要先于在采取简单的常规方法来解决内部问题，伴随方程和外部问题之间进行替代。尽管内部目标必须平滑，但外部物镜可能是非平滑的，但要承担缩减条件。我们证明，与伴随方程的精确和不精确的解决方案的梯度下降和前回向拆分的方法的线性收敛。我们在学习各向异性总变化图像denoising和图像反卷积的卷积内核方面表现出良好的表现。
We propose a new approach to solving bilevel optimization problems, intermediate between solving full-system optimality conditions with a Newton-type approach, and treating the inner problem as an implicit function. The overall idea is to solve the full-system optimality conditions, but to precondition them to alternate between taking steps of simple conventional methods for the inner problem, the adjoint equation, and the outer problem. While the inner objective has to be smooth, the outer objective may be nonsmooth subject to a prox-contractivity condition. We prove linear convergence of the approach for combinations of gradient descent and forward-backward splitting with exact and inexact solution of the adjoint equation. We demonstrate good performance on learning the regularization parameter for anisotropic total variation image denoising, and the convolution kernel for image deconvolution.
