论文标题
重新访问额外的分布式优化
Revisiting EXTRA for Smooth Distributed Optimization
论文作者
论文摘要
Extra是一种流行的方法，用于二进入分布式优化，并具有广泛的应用。本文重新审视了额外的。首先，我们通过改进的$ o \左（\ frac {\ frac {l}μ+\ frac {1} {1-σ_2（w）} \ right）\ log \ frac {1} {ε（1-σ_2（w）} \ right和complement $ commutions $ complemance $ complection和complemance $ complemance $ complectiities $ complection和complemance $ complectionition $ complection和complemance $ complection和complemance $ - $ l $ -smooth问题，其中$σ_2（w）$是重量矩阵$ w $的第二大单数值。当不存在强凸度时，我们证明了$ o \ left（\ left（\ frac {l}ε+\ frac {1} {1-σ_2（w）} \ right）\ log \ frac {1} {1} {1- = _2（w）} \ right）$ copplectiate。然后，我们使用催化剂框架加速额外并获得$ o \ left（\ sqrt {\ frac {\ frac {l} {μ（1-σ_2（w）}} \ log \ log \ log \ frac {l} {l} {μ（1-σ_2（w）} \ log frac \ frac \ frac \ frac \ frac \ frac \ frac \ frac \ frac convient和平滑的问题和$ o \ left（\ sqrt {\ frac {l} {ε（1-σ_2（w）}}} \ log \ frac {1} {ε（1-σ_2（w）\ right）$对于非刺激性convex One的复杂性。 Our communication complexities of the accelerated EXTRA are only worse by the factors of $\left(\log\frac{L}{μ(1-σ_2(W))}\right)$ and $\left(\log\frac{1}{ε(1-σ_2(W))}\right)$ from the lower complexity bounds for strongly convex and non-strongly convex分别问题。
EXTRA is a popular method for dencentralized distributed optimization and has broad applications. This paper revisits EXTRA. First, we give a sharp complexity analysis for EXTRA with the improved $O\left(\left(\frac{L}μ+\frac{1}{1-σ_2(W)}\right)\log\frac{1}{ε(1-σ_2(W))}\right)$ communication and computation complexities for $μ$-strongly convex and $L$-smooth problems, where $σ_2(W)$ is the second largest singular value of the weight matrix $W$. When the strong convexity is absent, we prove the $O\left(\left(\frac{L}ε+\frac{1}{1-σ_2(W)}\right)\log\frac{1}{1-σ_2(W)}\right)$ complexities. Then, we use the Catalyst framework to accelerate EXTRA and obtain the $O\left(\sqrt{\frac{L}{μ(1-σ_2(W))}}\log\frac{ L}{μ(1-σ_2(W))}\log\frac{1}ε\right)$ communication and computation complexities for strongly convex and smooth problems and the $O\left(\sqrt{\frac{L}{ε(1-σ_2(W))}}\log\frac{1}{ε(1-σ_2(W))}\right)$ complexities for non-strongly convex ones. Our communication complexities of the accelerated EXTRA are only worse by the factors of $\left(\log\frac{L}{μ(1-σ_2(W))}\right)$ and $\left(\log\frac{1}{ε(1-σ_2(W))}\right)$ from the lower complexity bounds for strongly convex and non-strongly convex problems, respectively.
