论文标题
$ l_1 $和$ l_2 $ - 非平滑凸随机优化问题的无梯度联合学习方法
Gradient-Free Federated Learning Methods with $l_1$ and $l_2$-Randomization for Non-Smooth Convex Stochastic Optimization Problems
论文作者
论文摘要
本文研究了凸随机优化的非平滑问题。使用平滑技术基于在考虑点上的函数值的替换，平均函数值在此点的中心的球（以$ l_1 $ -norm或$ l_2 $ -norm为单位（$ l_1 $ -norm或$ l_2 $ -norm））在此点，原始问题将降低到平滑的问题（其梯度的lipschitz常数是相对于球射线的lipschitz常数）。所使用的平滑度的一个重要属性是，仅基于对原始函数的实现，可以计算平滑函数梯度的无偏估计。提出获得的平滑随机优化问题是在分布式联合学习体系结构中解决的（该问题是并行解决的：节点是局部步骤，例如随机梯度下降，然后它们与所有人进行交流，然后全部重复）。本文的目的是基于当前无梯度非平滑优化的进步以及联合学习，无梯度的方法，用于解决联合学习体系结构中的非平滑随机优化问题。
This paper studies non-smooth problems of convex stochastic optimization. Using the smoothing technique based on the replacement of the function value at the considered point by the averaged function value over a ball (in $l_1$-norm or $l_2$-norm) of small radius with the center in this point, the original problem is reduced to a smooth problem (whose Lipschitz constant of the gradient is inversely proportional to the radius of the ball). An important property of the smoothing used is the possibility to calculate an unbiased estimation of the gradient of a smoothed function based only on realizations of the original function. The obtained smooth stochastic optimization problem is proposed to be solved in a distributed federated learning architecture (the problem is solved in parallel: nodes make local steps, e.g. stochastic gradient descent, then they communicate - all with all, then all this is repeated). The goal of this paper is to build on the current advances in gradient-free non-smooth optimization and in feild of federated learning, gradient-free methods for solving non-smooth stochastic optimization problems in federated learning architecture.
