论文标题
Lyapunov功能之间的连接用于某些优化算法和微分方程
The connections between Lyapunov functions for some optimization algorithms and differential equations
论文作者
论文摘要
在本手稿中，我们研究了具有阻尼，其离散化以及与加速优化算法的二阶微分方程家族的特性，用于$ m $ $ $ $ -Sronglongly的凸和$ L $ -SMOMOTH功能。特别是，使用\ emph {fazlyab et开发的线性矩阵不等式LMI框架。 al。 $（2018）$}，我们在分析上为Nesterov优化方法的两参数家族提供了分析（离散的）Lyapunov函数，从而可以完整地表征其收敛率。在适当的限制下，这种方法家族可以看作是二阶普通微分方程家族的离散化，我们通过LMI框架构建了（连续）Lyapunov的功能。可以通过研究其离散对应物的限制行为来获得连续的Lyapunov函数。最后，我们表明，诸如重球方法之类的ODE家族的大多数典型离散化都不具有Lyapunov函数，其属性具有类似于此处为Nesterov方法构建的Lyapunov函数的属性。
In this manuscript, we study the properties of a family of second-order differential equations with damping, its discretizations and their connections with accelerated optimization algorithms for $m$-strongly convex and $L$-smooth functions. In particular, using the Linear Matrix Inequality LMI framework developed by \emph{Fazlyab et. al. $(2018)$}, we derive analytically a (discrete) Lyapunov function for a two-parameter family of Nesterov optimization methods, which allows for the complete characterization of their convergence rate. In the appropriate limit, this family of methods may be seen as a discretization of a family of second-order ordinary differential equations for which we construct(continuous) Lyapunov functions by means of the LMI framework. The continuous Lyapunov functions may alternatively, be obtained by studying the limiting behaviour of their discrete counterparts. Finally, we show that the majority of typical discretizations of the family of ODEs, such as the Heavy ball method, do not possess Lyapunov functions with properties similar to those of the Lyapunov function constructed here for the Nesterov method.
