论文标题
在有向图上量化分散的随机学习
Quantized Decentralized Stochastic Learning over Directed Graphs
论文作者
论文摘要
我们考虑了一个分散的随机学习问题，其中数据点分布在有向图的计算节点之间。随着模型大小的大小，分散的学习面向一个主要的瓶颈，这是每个节点向其邻居传输大消息（模型更新），因此沟通负载很大。为了应对这种瓶颈，我们提出了基于分散的共识优化中的Push-sum算法的有向图上的量化分散的随机学习算法。更重要的是，我们证明我们的算法达到了分散的随机学习算法的相同收敛速率，并且对于凸和非核心损失的精确交流。数值评估证实了我们的主要理论结果，并说明了与确切通信方法相比的显着加速。
We consider a decentralized stochastic learning problem where data points are distributed among computing nodes communicating over a directed graph. As the model size gets large, decentralized learning faces a major bottleneck that is the heavy communication load due to each node transmitting large messages (model updates) to its neighbors. To tackle this bottleneck, we propose the quantized decentralized stochastic learning algorithm over directed graphs that is based on the push-sum algorithm in decentralized consensus optimization. More importantly, we prove that our algorithm achieves the same convergence rates of the decentralized stochastic learning algorithm with exact-communication for both convex and non-convex losses. Numerical evaluations corroborate our main theoretical results and illustrate significant speed-up compared to the exact-communication methods.
