论文标题
Nesterov加速梯度的联合学习
Federated Learning with Nesterov Accelerated Gradient
论文作者
论文摘要
联合学习（FL）是一种快速发展的技术，允许多个工人根据分布式数据集训练全球模型。常规FL（FedAvg）采用梯度下降算法，这可能不够有效。动量能够通过添加一个额外的动量步骤来加速收敛，并在集中式环境和FL环境中展示其好处，从而改善了情况。众所周知，Nesterov加速梯度（NAG）是一种更有利的动量形式，但尚不清楚如何量化NAG到目前为止的益处。这促使我们提出Fednag，该Fednag在每个工人中使用NAG以及聚合器中的NAG动量和模型聚合。我们提供了FedNag的详细收敛分析，并将其与FedAvg进行了比较。进行了基于现实世界数据集和痕量驱动模拟的广泛实验，表明FedNag将学习准确性提高了3-24％，并将总训练时间降低了11-70％，而在广泛的设置下的基准相比。
Federated learning (FL) is a fast-developing technique that allows multiple workers to train a global model based on a distributed dataset. Conventional FL (FedAvg) employs gradient descent algorithm, which may not be efficient enough. Momentum is able to improve the situation by adding an additional momentum step to accelerate the convergence and has demonstrated its benefits in both centralized and FL environments. It is well-known that Nesterov Accelerated Gradient (NAG) is a more advantageous form of momentum, but it is not clear how to quantify the benefits of NAG in FL so far. This motives us to propose FedNAG, which employs NAG in each worker as well as NAG momentum and model aggregation in the aggregator. We provide a detailed convergence analysis of FedNAG and compare it with FedAvg. Extensive experiments based on real-world datasets and trace-driven simulation are conducted, demonstrating that FedNAG increases the learning accuracy by 3-24% and decreases the total training time by 11-70% compared with the benchmarks under a wide range of settings.
